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CHAPTER  1 .  INTRODUCTION 
1.1.  ABSTRACT 


Suppose  there  are  given  k  >_  2  populations  observations 

from  population  are  normally  distributed  with  unknown  mean  ^  and 

Let 


2  2 

common  (known  or  unknown)  variance  *  a  (i  *  l,...,k) 


*  *lP[k]  denote  ranked  values  of  pj,...^.  In  this  thesis 
we  assume  throughout  that  both  the  numerical  values  of  and 

the  pairings  of  the  Wjjj » •  •  •  »U[k]  tfie  populations  are 

completely  unknown  (although  we  vary  the  distribution  from  normality) 
and  consider  the  problem:  estimate  some  (or  all)  of  yr j j . • . . *P[k) 
based  on  where  come  from  use  of  the  following 

single-stage  rule:  Take  n  independent  vectors  X.  *  (X  . , . . .  ,X,  .) , 

— j  LJ  KJ 

j  ■  l,...,n  (X^  denotes  the  jth  observation  from  tr^) »  for  each  popu¬ 
lation  compute  X^  ■  ^  X^/n  (i  *  l,...,k),  and  base  the  terminal 
decision  on  X^,...,)^.  (The  fixed  number  n  of  vectors  required  depends 
on  the  particular  problem.)  This  rule  has  been  used  in  many  instances 
of  statistical  decision  problems.  Applications  to  ranking  and  selection 
problems  are  noted. 

Let  .  ,<Xp.j  denote  the  ranked  Xj,...,X^.  A  natural  point 

estimator  of  p^j  is  X^  (1  _S  i  1  k),  and  its  bias  is  studied  when 
observations  from  tk  have  density  f(x  -  0^),  xeR,  where  the  location 
parameter  is  unknown  (i  =  l,...,k)  and  =  ^xf(x)dx<».  Upper  and 

lower  bounds,  and  L^,  are  derived  for  E^X^j  (1  _<  i  _<  k)  (p  denotes 
the  vector  (pj, . . . ,Pj,)) ,  and  condition  S(i),  sufficient  to  imply  that 
X^j  is  asymptotically  unbiased  as  n  +  »,  is  obtained.  When  i  *  k 
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(i  *  1),  IL(L^)  is  the  supremum  (iafimum)  of  E^X^j .  It  is  shown  that 

uniform  integrability  condition  C^(i)  implies  S(i).  Condition  C2  (which 

holds  if,  e.g.,  _£x2f (x)dx<®)  also  implies  S(i).  The  relationship  is 

C2  <■>  {Cj (1) , . . .  ,C^(k) }.  The  minimax |bias  (estimator  of  type  +  a 

is  found  for  certain  cases.  These  results  are  applied  to  the  case 

where  f(*)  is  the  normal  density,  and  a  uniform  integrability  argument 

shows  that  IL  and  are  the  supremum  and  infimum.  It  is  noted  that, 

for  the  location  parameter  case,  X^j  is  strongly  consistent  for 

(1  1  i  1  k) ;  applications  are  noted.  Bounds  are  obtained  on  the  mean 
_  2 

squared  error  E^(X^  -  (1  ^  i  ^  lc) ,  also  for  the  location 

parameter  case.  For  the  case  when  f(«)  is  the  normal  density  these 
bounds  are  evaluated,  and  intervals  in  which  the  supremum  and  infimum 
of  the  mean  squared  error  lie  are  determined. 

Maximum  likelihood  estimation  of  (u jjj , . . . ,p )  based  on 
X^j,...,X^j  is  studied.  It  is  shown  that  any  critical  point  for 
this  problem  is  a  solution  of  a  system  with  derivatives  taken  for 
U  e  r  {y:  y  ^  t  ...  /  y^  }  if  houndary  points  are  con¬ 
sidered  solutions  and  that  (X,...,X)  with  X  *  ^X[i]  +  +  X[k]^^ 

is  a  critical  point.  The  nature  of  (X, . . . ,X)  is  completely  determined, 
and  w.p.  -*■  1  as  n  -*•  00  it  is  a  saddle  point  (unless  y^j  =  ...  *  y^, 
in  which  case  it  may  be  a  relative  maximum) .  Some  results  on  the  form 
of  the  maximum  likelihood  estimator  (WLE)  for  k  >_  2  are  given,  while 
for  k  *  2  the  MLE  is  found  explicitly.  MLE's  for  non-1-1  functions  are 
discussed,  and  a  concept  of  iterated  MLE's  (IMLE's)  is  introduced  and 
discussed.  The  generalized  MLE  (GMLE)  introduced  by  Weiss  and  Wolfowitz, 
which  has  a  certain  optimality  property,  is  found  to  be  X^j , . . .  ,Xpcj , 
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which  has  desirable  large  sample  concentration.  It  is  shown  that  there 
is  not  just  one  GMLE  but  rather  a  whole  class  of  CMLE's,  and  for  k  ■  2 
the  MLE  is  shown  to  be  in  this  class  along  with  X^,  It  is  shown 

that  for  our  problem  (and  others)  a  GMLE  (if  one  exists)  is  equivalent 
to  the  maximum  probability  estimator  (MPE)  introduced  by  Weiss  and 
Wolfowitz,  if  the  latter  is  "good." 

Confidence  interval  estimation  of  y^,...,  y^  is  discussed,  and 
upper  and  lower  intervals  on  y^j  (1  <  i  <  k)  are  found,  along  with 
their  maximal  overprotection,  for  location  parameter  populations. 
Generalizing  a  result  of  Fraser,  it  is  shown  that  exact  upper  intervals 
satisfying  mild  conditions  do  not  exist. 
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CHAPTER  1 .  INTRODUCTION 
1.2.  OUTLINE  OF  THE  THESIS 


In  Section  1.1,  we  have  given  an  overview  of  the  problem  considered 
below  and  of  the  results  obtained,  and  in  Section  1.3  we  make  specific 
definition  of  the  problem  considered  and  introduce  various  notations. 

In  the  present  section  we  outline  briefly  the  contents  of  the  various 
chapters . 

Chapter  2.  The  problem  of  point  estimation  is  considered  for  a 
location  parameter  family,  and  the  bias  of  certain  natural  estimators  is 
studied;  a  minimax  estimator  is  found  for  certain  cases.  These  general 
results  are  examined  in  the  normal  density  case,  for  which  additional 
results  are  obtained. 

Chapter  3.  The  problem  of  strong  consistency  is  considered  for  a 
location  parameter  family,  and  applications  to  value-estimation  and 
Bayesian  statistics  are  noted. 

Chapter  4.  For  a  location  parameter  family,  bounds  are  obtained  on 
the  mean  squared  error  of  certain  natural  estimators.  These  results  are 
examined  in  the  normal  density  case,  and  additional  bounds  on  the 
infimum  and  suprenum  of  the  mean  squared  error  lead  to  intervals  on 
these  two  quantities. 

Chapter  5.  Maximum  likelihood  estimators  are  studied  for  the  normal 
density  case.  A  concept  of  iterated  maximum  likelihood  estimators  is 
introduced  and  discussed.  Generalized  maximum  likelihood  estimators  and 
maximum  probability  estimators  are  fonnd. 
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Chapter  6.  The  problem  of  interval  estimation  is  formulated.  For 
a  location  parameter  family  upper  and  lower  intervals  are  found,  and  it 
is  shown  that  exact  upper  intervals  satisfying  mild  conditions  do  not 
exist. 


CHAPTER  1.  INTRODUCTION 


1.3.  PROBLEM  DEFINITION  AND  NOTATION 


Consider  the  set-up 

Given  k(^2)  populations  ^,...>1^  such  that  observations  from 

(1.3.1)  population  tt.  are  normally  distributed  with  unknown  mean  y^  and 

2  2 

common  (known  or  unknown)  variance  =  o  (i  ■  l,...,k), 

and  the  following  rule. 

RULE:  Take  n  independent  vectors  X^.  »  (^.....X^), 
j  *  l,...,n,  where  X„  denotes  the  jth  observation  from  the 

ith  population  ir. .  For  each  population  form  the  sample  mean 

(1.3.2)  1 

(1.3.3)  X.  »  l  X../n  (i  »  l,...,k), 

and  base  the  terminal  decision  se’ely  on  the  statistics 


(This  rule  has  been  utilized  under  set-up  (1.3.1)  in  many  instances  of 

statistical  decision  problems.)  Make  the 

DEFINITION:  Let  Urn<-**<Pri,i  denote  the  ranked  values  of 

(1.3.4)  llJ  lKJ 


We  assume  throughout  that  both  the  numerical  values  of  and  the 

pairings  of  the  y^j » •  ■  •  »y[k]  populations  are  com¬ 

pletely  unknown  (although  we  vary  the  distributional  requirements  from 
those  of  set-up  (1.3.1))  and  consider  the  problem:  estimate  some  (or 
all)  of  y - 


ur 

stage  Rule  (1.3.2) . 


,  y  based  on  the  statistics  provided  by  the  single- 
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Consideration  has  been  devoted  in  the  literature  to  what  are  called 
"ranking  and  selection"  problems.  Since  several  of  the  proposed  pro¬ 
cedures  in  that  type  of  statistical  decision  problem  use  Rule  (1.3.2) 
(e.g.,  those  of  Bechhofer  (1954),  Gupta  (1956),  (1965),  and  others),  and 
since  one  will  often  wish  to  estimate  as  well  as  select,  we  will  briefly 
describe  such  problems  and  will  refer  below  to  uses  of  our  results  in 
such  problems. 

A  simple  example  of  such  a  problem  is  that  of  selecting  the  popula¬ 
tion  (or,  one  of  the  population.  )  associated  with  the  ith  smallest  mean 
(1  i  i  i  10;  this  is  called  one's  goal.  (Much  more  general  goals  have 
also  been  considered.)  Typically,  a  probability  requirement  is  made  and 
a  procedure  is  given  (which  tells  how  to  sample,  when  to  stop  sampling, 
and  what  terminal  decision  to  make).  The  probability  requirement  affects 
one's  sample  sizes,  since  the  more  stringent  one's  probability  require¬ 
ment  vis-a-vis  achieving  the  goal,  the  more  sampling  one  must  perform. 

In  Rule  (1.3.2),  only  the  fixed  number  n  of  independent  vectors  required 
depends  on  the  particular  (goal,  probability  requirement,  procedure) 
structure  on  hand.  (We  note  that  Rule  (1.3.2)  has  some  optimal  proper¬ 
ties.  See  Hall  (1958),  (1959);  Bahadur  and  Goodman  (1952);  Lehmann 
(1966);  and  Eaton  (1967).)  Of  course  the  various  structures  use  the 
statistics  in  quite  different  manners,  and  not  all  structures  use 
Rule  (1.3.2);  e.g.,  the  nonparametric  procedure  of  Bechhofer  and  Sobel 
(1958),  the  closed  sequential  procedure  of  Paulson  (1964),  and  the  open 
sequential  procedure  of  Bechhofer,  Kiefer,  and  Sobel  (1968)  do  not. 

We  will  make  use  of  the  following  definitions  and  notation. 

DEFINITION:  For  any  set  S,  let  v(S)  =  cardinal  number  of  S. 

(1.3.5) 

(If  S  is  a  finite  set,  then  v(S)  is  the  number  of  elements  in  S.) 
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(1.3.6) 


(1.3.7) 


(1.3.8) 


(1.3.9) 


(1.3.10) 


DEFINITION:  Let  R  *  (x:  -»<x<»)  and  let  P*  »  {x:  x>0) 
— ■  -  -■  —  • 


DEFINITION:  For  6  e  R+,  let  n6(atb,c, . . .)  »  {(p  f . . . ,  j^) : 

“  u  j  j  —  g  R  *  1»  •  •  •  ik)  i  •  ■  i  ftrc  held 

fixed}.  (In  general  a,b,c,...  will  be  several  of 

w[l]*****w[k]^ 


DEFINITION:  Let  <dlfc(6)  «= 
l°EM(p[k])  =  (y(k],,,,,u[k]) 


and 

be  vectors  of  k  components. 


DEFINITION:  Let  Xr, , <. . . <Xr.  ,  denote  the  ordered  X. 
-  [1]-  “  [k]  i 

(i  *  l,...,k).  (We  disregard  the  possibility  of  ties, 
which  occur  w.p.  0  in  the  cases  considered  below.) 


DEFINITION:  If  a  random  variable  (r.v.)  X  is  normally 

2 

distributed  with  mean  u  and  variance  o  ,  we  shall  say 
X  is  N(u,o2). 

Denote  the  N(0,1)  distribution  function  (d.f.)  and 
density  function  (fr.f.)  by  ♦(")  and  $(•).  respectively; 
i.e.,  let 

*(x)  =  j[<Ky)dy  (x  c  R), 

1  2 

1  ~2^ 

$  (y)  »  -  e  (y  e  R) . 

/2'tT 


DEFINITION:  Let  F  and  f  be  the  respective  d.f.  and  fr.f.  of 
observations  from  an  arbitrary  univariate  location  parameter 


(1.3.11) 


family;  i.e., 
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(1.3.12) 


(1.3.13) 


(1.3.14) 


F(x)  -  _?f(y-9)dy 
f  has  the  form  f(y-e) 
where  0  is  fixed,  0  e  0  c  R. 


(x  e  R),  and 

(ye  R), 


DEFINITION:  fitf)  *  (y:  v{1]  t  u[2]  t  ...  *  yfk] }. 


DEFINITION :  If  yefi(^) ,  let  X  ^  denote  the  sample  mean 

produced  by  the  population  associated  with  yr.,  (i  =  l,...,k) 

l1! 


DEFINITION:  If  there  is  at  least  one  break  in  the  string  of 
inequalities  t  ...  t  ^[]<]»  then  the  situation  is  that  we 
have  £(l<£<k)  groups  of  equal  parameters 

“[1]  "  '  "[^l  *  w[ij+l]  *  *  uti2] 

*  *  "[ij.!*!]  ’  '  “[k] 

with  ij,...,i  j  integers 

(0  =  i  <l<i.<i_<. . ,<i  ,<k-l<i  =  k), 

o—l  Z  £-1—  £ 

and  we  let 

be  the  ranked  values  of  the  sample  means  from  the  population(s) 

associated  with  parameter  y,.  ,  (j  *  0,...,£-l). 

llj+lJ 


DEFINITION 


:  Let  be  the  symmetric  group  on  k  elements,  i.e. 


(1.3.15) 


(a:  a  =  (a(l) , . . .  ,a(k))  is  a  permutation  of  (!,...,]:)  }• 


CHAPTER  2.  POINT  ESTIMATION:  BIAS 
2.1.  BIAS  OF  A  NATURAL  ESTIMATOR  OF  (l<i<k) 
FOP.  A  LOCATION  PAPAMETER  FAMILY 


Consider  the  set-up 

Given  k(>2)  populations  such  that  observations  from 

(2.1.1)  population  tk  have  fr.f.  f(x-0^),  x  c  R,  where  the  location 
parameter  9..  is  unknown  (i  =  l,...,k). 

We  make  the 

(2.1.2)  ASSUMPTION:  The  fr.f.  f  is  such  that  Ef  =  _£xf(x)dx<°°. 


so  that  we  may  talk  of  (or  of  , . . .  »p[j,j)  •  denote  the 

ranked  values  of  the  location  parameters  6 ^ , . . . , 0^  by  0^.i*  •  < 


Then  since 


(2.1.3) 


£xf(x-a)dx  »  _/(x+a-b)f (x-b)dx  =  £xf (x-b)dx- (b-a) 


_£xf (x-b)dx 


(a<b;  a,b  e  0) , 


the  population  associated  with  is  precisely  the  population  associ¬ 

ated  with  (i  =  1 , . . .  ,k) .  Also, 

(2.1.4)  _£xf(x-0)dx  =  _/xf(x)dx  +  0  »  +  0 

where  is  the  mean  of  f  when  0=0. 

We  will  now  study  estimation  of  (l<i<k)  when  set-up  (2.1.1) 

obtains,  Rule  (1.3.2)  is  used,  and  the  pairing  of  with 

y jlj » • • • »u is  completely  unknown  (see  Chapter  1).  denote  the 

densities  of  X..-0.  and  X..  by  fv  a  and  fv  ,  respectively.  Since 
111  ij  a.  .-o.  X. . 

ij  i  ij 
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(2.1.5) 


fx  e  (y)  "  fx  <y+ei>  "  f  (Cy+e  )-e  )  =  £(y), 

ij  i  ij 


it  follows  that  X^-0^  does  not  depend  on  6.  (i  =  l,...,k) 


(2.1.6) 


DEFINITION:  Gn(y|f)  -  PlUX.j-0.)*. .  .  +  (X.n-0 . )  }/n  <  y] , 

en(ylf)  '  -^-G„cy|«- 


For  i  =  1 , . . .  ,k, 

P  [Xjlx]  =  P[Xn  +  ...+Xin<nx]  =  P[(Xil-0.)  +  ...  +  (Xin-0i)<n(x-0.)] 

(2.1.7) 

*  Gn(x-0.|f). 

We  now  determine  several  d.f.'s  and  fr.f.'s  which  we  will  use  in  later 
sections. 

k 

THEOREM:  Fv  (x)  =  n  G  (x-0.  |f)  (x  e  R) , 

X[k]  i=l  n  1 

(2.1.8)  k 


fv  (x)  *  I  I"  f  n  G  (x-0.  | f ) ]  g  (x-0.  id  (x  e  R) 
X[k]  j«lL  li?jl  1  1  1 


Proof: 


F^r  (x)  =  P(max(Xj ....  ,Xk)  £  x]  *  P[Xj  <_  x, . . .  ,X^  <_  x] 

[k] 

k 

a  P[X.  <  x]...P[X,  <  x]  =  n  G  (x-0.|f). 

1  -  K  -  i=1  n  i 

The  expression  for  fy-  (•)  follows  upon  differentiation  of  F^  (•)» 

X[k]  X[k] 

utilizing  the  chain  rule  (see,  e.g.,  Kaplan  (1952),  p.  86,  (2-26))  and 

the  fact  that  G'(y|f)  =  — G  (y | f )  =  c  (y | f)  (see,  e.g.,  Fisz  (1963) 

n  '  dy  n  n 

p.  35;  or  Parzen  (1960),  p.  169). 


A 


COROLLARY:  E  Xr.  ,  =  /x f7  (x)dx 

v  ■“  ^[k] 

(2.1.9)  k  - 

-l  /xSn(x-e|f) 

J=1  J 


nGn(x-ei|f) 


dx. 


A  possible  estimator  of  when  set-up  (2.1.1)  obtains  and 
Rule  (1.3.2)  is  used  is  X^j  (i  =  1 , . . .  ,k) ;  we  now  study  its  expectation 
and  bias.  (Although  quantities  such  as  EyX^j  depend  on  the  unknown 
u  e  fi0»  this  dependence  will  sometimes  be  suppressed;  e.p.,  we  will 
write  EX^j  for 


LFfffiA:  If  X  and  Y  are  independent  r.v.'s  with 
(2.1.10)  FXM  =  ptx±x3  ^  P[Y^x]  =  Fy(x)  (x  e  R) , 

then  EX  >_  EY. 

Proof :  A  peometrical  proof  of  this  lemma  can  easily  be  given  using, 
e.g.,  Exercise  2.5  of  Parzen  (I960),  pp.  211-212,  "A  peometrical  inter¬ 
pretation  of  the  mean  of  a  probability  law.” 


THEOREM:  For  i 

(2.1.11) 

(t  -  1  .....jo. 


1 , . . . ,k  and  x  e  R, 


Fy  (x)+  as  v  + 

[i] 


Proof :  Fix  £(1  ^  k).  For  i  =  1, . . .  ,k  and  x  e  R, 

FY  (x)  =  P  [Xr.,<x]  =  P  [The  ith  smallest  of  X,,...,X.  is  <x] 

A^j  u  l  x.  j  u  i  k  — 

=  Py[At  least  i  of  Xj,...,X^  are  <_  x] 

=  Py[X£lx  and  at  least  i-1  of  Xj , . . .  ,X£1  ,Xfc+1 , . . .  ,X^  are  <_  x] 

+  P  [X  >x  and  at  least  i  of  X, ,,..,X.  .  ,X„  .,..., X,  are  <  x] 

■  Py [X£<x] Py [At  least  i-1  of  , . . . ,Xft-1,X£+1 , . . . ,Xk  are  <  x] 


■  nn(x'eJf)pyrAt  least  i_1  of  >V"-’Vi*’Vi'---»xic Rre  ixi 

♦  [l-Gn(x-et|^)TP  [At  least  i  of  Ij,  •  •  •  >'xz . Xk  are  5-xl  • 

Therefore, 

-d  -  F--  (x)  =  d_  F-  (x)  dei 

dp£  Til  H0l  XTi]  f,M£ 

■  '(Tnfx'eJf^fVAt  least  i_1  of  xr,,,,y£-rxx+i»,--,xk  are  -x^ 

+p  (x-eJf)P  [At  least  i  of  X.,...,X  .  ,X  X,  are  <x] , 

It  Jc.  Jj  1  JL~1  Jc^l 

which  is  <0  iff 

P^FAt  least  i  of  Xj, . . .  ,X^+J . yk  are  <xl 

<P  T*t  least  i-1  of  Xj, . .  .Jz  j  ,X£+J , . . .  ,XV  are  <xl  . 

nFFP'lTieN?:  For  &  »  1,7,1,...  let  h  (?  )  Ke  the  expectation 

-  £  P 

of  the  naxinum  of  £  independent  r.v.'s  each  havin"  fr.f.  gn(x); 

and  let  h'(p  )  be  the  expectation  o^  the  minimum  of  £  in^e- 

(2.1.12)  1  n 

pendent  r.v.'s  each  having  fr.f.  g^fx),  i-e., 

\(pn)  s  1«nfv)^v, 

h£(?n)  *  _7y^Fi-f;n(y)l£'1PT1(y)f1y. 

Tie  following  is  well-known: 

IF " V :  If  rn(x)  is  symmetric  about  x  =  0  then 

(2.1.15) 

W  •  -W- 

(2. 1.13a)  'p,FOnrM:  If  G  (x)  <  1  for  all  x,  then  lim  b  (g  )  *  +». 

-  n  £-*»  £  n 


Proof:  Py  (2.1.12), 
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AJy[Gn(y)]*"1gI,Gr)d>r  2  h£^n)  =  £_1gn(y)dy 

■  ^y[Gn(y)l£’Vn(y)dy  +  5£yfGn  (y)  1  £"1gn(y)dy 

>.  ^y[Gn(y)]A1«n(y)dy  +  4fGn(°)]£"1_£/BBfy)dy* 

Thus,  since  7yg  (y)dy  <  «  and  lim  £a*  =  0  (0<a<l),  by  taking  the  limit 
"  £-*» 

as  £-*»  we  obtain 

lim  hn  (p.  )  =  lim  £?y[G  (y)l^'V  Cy)dy. 

£-*»  £V,V  £-tos  o7 1  n^-^J  r,n  1 

However,  for  any  '1  >  0, 

0  <  ^y[cr(y)]*_1gn(y)dy  <  ^yfGnC)l  £’1pnCy)dy 
=  £[Gn(M)]£_1  J(y£n(y)dy->0  as  £-*». 

Choosing  M  >  1,  and  since  G  (M)  <  1  for  any  ’I,  we  find  that 

USW  ■  Ji  *ft'tn,,(y)]1'1f>n(y)<ly>|ir  •«7fCn(y)]*'1Pn(y)dy 

■  Si2M(Vy,,1|M *  lz  =  M. 

Since  M  >  1  was  arbitrary,  the  theorem  follows. 

LEMMA :  If  Yj , . . .  ,Y£  are  independent  r.v.'s  each  having  d.f. 
Gn(x-0),  then 

(2-1.14)  K  max{Yj .... ,Y£)  =  e  ♦  h£(pn), 

T  min(Y1,...,Y£}  =  6  ♦  b£(pn)  . 

Proof:  Since  YJias  d.f.  C-n(x-e),  Yj  -  e  has  d.f.  Gn(x)  (i  =  1,...,£) 
by  (2.1.7) .  Thus, 

E  min  fY,,...,Yt)  •  E  ffiJS  {(Y,-e)*e . (Yt-e).8) 

=  6  *  E  Rin  (Y  -9 . Y,-e)  .  e  .  !VPn) 

iw- 
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(2.1.15) 


Proof : 
raising  u 


THEOREM :  For  any  i  (1  <  i  <  k) 


sup{iy[1]:  „  £  n0(Mfi])) 


<  E 


(2.1.16) 


xri 

^u[l]  *' '  *  *w[i-l]  ,lJfi]»yfi+l]  * '  • '  ,p[k]) 


(p [ j]  >  •  •  •  >R  ^  [i]  *+00’ " "  *  *+0°^ 


'  6[i]  *  W  '  ufi]  •  Ef  *  M"J. 


x  n 


and 


(2.1.17) 


inf{FyX[i] :  V  e  Vg[i]^ 

>Fr  X[i] 

(yfl],...,Pji_lj,ufi],yri+i] 

“  (  “  oo,  •  •  .  ,  “  oo> 


. u[k]) 


kjii-tii . “m 

k-i+1  times 


) 


efi]  +  Nl-i+l^n* 


w[i]  "  Ff  +  bk-i+l^n5. 


where  the  configurations  of  the  vector  (wj-j  j , . . . ,  Rp^j)  "’hich 
involve  values  ♦«  are  viewed  as  a  situation  elininatinp  the 
populations  with  mean  values  ^«>  from  contention  for  ith 
highest  sample  mean.  (The  case  i  =  k  in  (2.1.16)  and  the 
case  i  =  1  in  (2.1.17)  involve  no  such  eliminations.) 
y  Lemma  (2.1.10)  and  Theorem  (2.1.11),  we  increase  hy 

j  (i . j  ■  l,...,k).  Mow, 

9[i]  *  "k-i.l'Cn’ 


=  n 


(Smal lest  of  (X^j-0^j)  +  6|-.j,...,(X£j^-8|-^i)  +  9|-j-j} 


<  g  { i th_  smallest  of  » •  •  •  *^(i)  *  *  *  *  ’^(k)  ^ 

"  =tU(H . “[i-l],IJ[i],l,[i) . "[i)1 

<_  { ith  smallest  of  X^^  >  •  •  •  *X(i)  *  *  * "  *X^j  )* 

=  E^{ ith  smallest  of  X^j^ , . • . jX^^ *X^i+ij *  *  *  * ’X^k) ^ 

<  j-  ^ ith  smallest  of  X^j » • *  * *^(i+l) *  *  *  * *^fk) ^ 

u  . >,[iJ*l,[l]*iJCi*l] . “[k]) 

<  B  {larBest  of  t5rci)'e[ilJ*eri] . C5rCi)'etil)+eri]> 

=  0  r . 1 +h.  (g  ) . 

[1]  iV6n' 

(Note  that  for  our  purposes  here,  the  ties  in  Definition  (1.3.14)  should 
be  broken  in  an  arbitrary  manner.)  Upon  takine  the  desired  supremum  and 
infimum,  the  theorem  follows. 


COROLLARY:  For  any  i  (1  i  i  <_  k) 

(2.1.19)  yfij  +  ^k-i+l^'V  -  E/[i]  -w[i]  +  (hi(pn,“Ef)' 
Thus,  (1)  is  asymptotically  unbiased  (as  n-n»)  as  an 

estimator  of  yr.,  if 

[i] 

(2.1. 18)  jhi(l!n)<5f  as  ■  and 

1  (2.1.20)  < 

lhk-i*l(8n)*Ef  as  : 

(2)  if  the  left  and  rinht  members  of  (2.1.19)  are  the 
infimum  and  supremum  of  E^X^  (respectively)  then 
X^j  is  asymptotically  unbiased  (as  tv*»)  iff 


(2.1.20)  holds. 
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With  Corollary  (2.1.18)  as  motivation,  we  will  now  study  the  questions 
of  (i)  when  (2.1.20)  holds  and  (ii)  when  the  inf  and  sup  above  achieve 
the  bounds  of  (2.1.19). 


THEOREM: 

(2.1.21)  P[k]  ^  V[k]  -  sup(EgX[k]:  P  e  noCw[k])}  “  v[k]  * 

w[l]  +  fhk^n)-Ef)  “  inf{E/[i]:  w  e  R0(lJ[l])}  -  EpX[l]  -  yfl]' 
Proof:  The  lower  bound  for  E^X^j  (the  upper  bound  for  E^X^)  follows 
from  the  fact  that  hj(?n)  »  (that  hj(gn)  *  E^)  •  The  equality  for  the 
sup  for  E^Xjkj,  and  for  the  inf  for  E^X^,  follow  easily  from  Theorem 
(2.1.15)  and  the  first  sentence  of  the  proof  of  Theorem  (2.1.15).  Note 
that  they  are  actually  attained  at  WgjjCu  rj.j)  an<*  wEf.(wjjj)»  respectively. 


From  Assumption  (2.1.2),  it  follows  that  independent  r.v.'s  with 
fr.f.  f  obey  the  Law  of  Large  Numbers,  so  that  (cf.  (2.1.7))  as  n+<», 
for  any  i  (1  1  i  1  k) 


Gn(y|f)  \ 

i  I  n,y  <  Ef 

(2.1.22)  [Gn(y|f))1  V  Gw(y| f)  =  \ 

l-[l-Gn(y|f)]k-i+1j  '1,yLEf 

since  (2.1.4)  is  true.  Each  of  the  convergences  indicated  in  (2.1.22) 
is  weak  convergence;  i.e.,  converges  weakly  to  F  iff  F^-JF  on  the 
continuity  set  of  F.  It  is  not  obvious  that  it  is  then  the  case  that 
(2.1.20)  holds,  i.e.,  that  for  any  i  (1  <  i  <  k). 


(2.1.23) 


hi(gn)  ‘.^/[Gntylf)]1) 


k-i-1 


K-Ul^r?  “  -^dy{l'[l'Gn(y|f)i  "  ) 


s  -*•  Ef  as  n- 


If  we  make  the  following  definition  (cf.  Lofeve  (1963),  p.  182) 


DEFINITION:  If  g(*)  is  a  continuous  function  and  Fn(0  is  a 
d.f .  (n  >_  1) ,  we  say  |g  |  is  uniformly  integral? le  in  Fn  if 

/  |g|dF  -K)  uniformly  in  n  as  c  -*»with  m-*«;  i.e.,  if  (for 

(2.1.24)  |x|>c  n  n 

—  m 

any  e  >o)  there  is  an  m  such  that  for  m  >  m  we  have 
I  x  I  >c  I  ®  I  <  e  ^or  n  (w^ere  c^-*-00  as  n-»-oo) , 

i  i —  ui 

then  we  may  use  the  following  theorem  (cf.  LoSve  (1963),  p.  183, 
Theorun  A.(ii)) 

THEOREM :  If  converges  weakly  to  F  (a  d.f.)  and  |g|  is 
(2  1  25)  unif°rmly  integrable  in  F^,  then 

/gdFn  -  /g dF 


to  immediately  state  the 

THEOREM:  For  any  i  (1  <  i  <  k) ,  (2.1.20)  holds  if  |y|  is 

(2. I-26)  i  k.i+1 

uniformly  integrable  in  (G^ (y  |  f )  ] 1  and  1  -  [  1  (y  | f )  1 

Proof :  This  follows  from  (2.1.22),  (2.1.23),  and  Theorem  (2.1.25). 


(2.1.27) 


00  * 

THEOREM:  If  (2.1.20)  holds,  then  _/ydy{ [Gn(y |f)]1>-HEf,  and 
then  £ydy{[Gn(y|f)]xHEj,  _£|y|dy{  [G^ (y  | f)  1  X)-^E~  with 
F.f  =  E+  -  E~.  (F~  =  lim  .?|y|dy{[Gn(y|f)]i};  E*  similarly.) 

For  any  i  (1  <_  i  <_  k),  (2.1.20)  holds  only  if  (as  n+») 

r2ET,  0  <  E.  <  M 
f  |y|d  UG  (yjf)]1}  -*•  i 

|y|>M  [2E|,  -M<  Ef  <  0. 

Note  that  |y|  is  unifomly  integrablt;  in  [Gn(y | f ) J  means 
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E^  =  0  if  E^.  is  non-nerative  (F£  =  0  if  F^  is  non-positive). 

k  “  i  ♦  1 

A  similar  result  holds  with  respect  to  { 1-[1-Gn(y|f)]  "  A). 

Mote  that  and  may  depend  on  i. 

Proof :  Suppose  0  <  E^.  <  M.  By  the  Helly-Rray  Lemma  (see,  e.g.,  Lofeve 
(1963),  p.  180), 


M 

/ydyUG^ylf)]1}^  ' 
o 

/  yd  ( [G  (y  |  f )  ] 1  HO  . 
-M  y  n 


as  n-*»  . 


Now,  letting  r>-»®  in 

tcn(y I i J  -  iydyd^C/lf)]1) 


M  .00 

♦  iydy([cn(y|f)]1)  *  ^ydyl^Cylf))1) 


we  obtain 

Ef  ■  -E7<Ef  *  A52  7yd  f  [G^tylOJ1). 

M 

00  i  — 

so  that  /yd  {  [G  (y|f)]  }-*Ef  as  n-*»,  and  thus 
M  y  n  * 

•  -H  •  i 

/|y|dj  [G  (y  |  £)  ]  >  «  -./yd  {[Gn(y|f)]1>  ♦  /yd  ( fGn(y|f)]1} 

jy|>M  y  1  M  1 

-►(E^-0)  ♦  E^  =  2E^  as  n-*»  . 


The  case  -M  <  F.^.  <  0  follows  in  a  similar  manner.  The  result  for  E^  =  0 
follows  from  the  eolation  E^.  =  E*  -  E^  and  the  Helly-Bray  Lemma. 


We  have  thus  seen  that  although  a  certain  uniform  integrability 
condition  is  sufficient  for  (2.1.20)  to  hold  (Theorem  (2,1.26)),  it  is 
not  clear  that  it  is  necessary  for  (2.1.20)  to  hold  (Theorem  (2.1.27)). 

V'e  will  now  exhibit  a  condition  (simpler  than  that  of 
Theorem  (2.1.26))  under  which  (2,1.20)  holds.  Fix  i  (1  <  i  <  k),  let 


,...,Z.  each  with  fr.f.  f(-) 
'  ’  jn  ' 
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(j  =  and  suppose  EZ.  =  y  (say)  exists.  Me  wish  to  know  when 

J  ** 

(as  n-**>) 

E!S  • 


(2.1.28) 


THEOREM:  If  E|Z^-y|->0  (as  n-*»)  (j  =  l,...,i),  then  (as  n-*«) 
E  ISfn  (Z1,...fZ,)-ii  . 


Proof : 


max 

mm 


(Z 


■  £1^2  +  lZl'Z2l 

1*  2J  ‘  2  "  - 2 -  * 


so  that 

E  min  (Zj.y  ”  “ 

However  (since  |a|-|b|  <  |a-b|  for  a,b  e  f?) 

|zrz2|  1  (Zj-Iil  ♦  |z2-u|, 

and  thus  (as  n-^°)  by  the  hypotheses  of  the  theorem  E  |Z.-Z- |*0.  The 
result  for  k  >  2  follows  by  induction. 


Although  it  can  be  proven  (see,  e.g.,  Loeve  (1°63),  p.  157,  d.) 
that  E  IZj-u  |-K1  implies  that  E  |z^  |-*-|u  [,  it  is  not  clear  v;hen  the  converse 
is  true.  In  our  situation,  we  would  like  to  know  when  EZj^p  implies 
E|Zj-u|-*0  (i.e.,  for  which  f  ( *)  *  s  this  is  the  case). 

(2.1.29)  THEOREM:  If  var(Zj)-0  (as  n-*=°)  then  E  fZj- p  |-*n . 

1  ir 

Proof :  This  follows  directly  from  the  fact  that  (E  [X  |  )  r  is  a  non¬ 
decreasing  function  of  r  >  0  for  any  r.v.  X  (see,  e.g.,  Lodve  (1963), 
p.  156,  c.). 

(2.1.30)  LEMMA:  Var  (Z, ) +  0  iff  /x2f(x)dx<«. 

A  “00 


1 


Var(Zj)  =  J-  var(Zu)  =  ^  (_£x2f(x)dx  -  (_/xf(x)dx)2). 


Proof : 
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These  results  on  the  satisfaction  of  (2.1.20)  of  Corollary  (2.1.18) 
may  be  summarized  as  follows. 

SUMMARY:  For  any  i  (1  _<  i  _<  k),  is  asYr  ’  Mcally 

unbiased  (as  n-*»)  as  an  estimator  of  y^  if 

(1)  |y|  is  uniformly  integrable  in  [Gn(y  If)]1 

and  1-[1-Gn(y|f)]k'i+1, 


(2.1.31) 


or  if 

(2)  _£c2f(x)dx<». 

(Note  that  (1)  holds  if,  as  is  often  the  case,  f(.)  is 
concentrated  on  a  bounded  set  in  R.) 


For  reasons  noted  above  Lemma  (2.1.10)  it  was  reasonable  to 
study  the  expectation  and  bias  of  as  an  estimator  of  y^j 
(i  =  l,...,k)  in  our  context..  l"ith  Corollary  (2.1.18)  as  motivation, 
we  note  that  estimators 

(2.1.32)  Xj.j  ♦  a  -  n£  i  *  i  W  -  V 

(correction  of  X^j  by  adding  a  constant)  may  be  preferable  to  X^  in 

certain  contexts.  If  positive  (negative)  bias  is  very  undesirable, 

one  may  use  a  *  h.(g  )  -  E-  (a  *  hJ  .  .  (g  )  -  E.)  and  obviate  its 
i  n  t  k-1+i  n  t 

possibility.  If  one's  preferences  on  bias  are  more  complicated,  one 
might  even  remove  the  restriction  h'  .  .  (g  )  -  E-  <  a  <  h.(p  )  -  E,.. 
(Note  that  this  restriction'frakes  sense"  since  (see  (2.1.14)  for 
notation) 

h'  (gj  *  E  rin(Y )  <  EY  <  F  nax(Y  ,...,Y„  )  *  h„  (pj.) 


a  ‘n 
l 


1  ~ 


a2  v^n 


Note  that,  for  certain  f(*)'s,  information  about  the  distribution 
GJ1(,|f)  will  be  available  for  use  in  determining  C.°n)  and  h^  (Pn) 


(1  <  i  <  k) .  For  information  and  references  see  Peitsma  (1963). 


(2.1.33) 


Proof : 


For  (2), 

a 


since  (for 


THEOPEM:  Fix  i  (1  _<  i  <_  k) .  Suppose  that  the  sup  and  inf 
of  (2.1.19)  achieve  the  bounds  of  (2.1.19).  Then  we  minimize 

IflclX  (r  V  o  , ,  'i 

min  CE,,Xr - !  -  a  -  Fri1) 


0) 


F  e 


7'[i] 
no(u[i]' 


[i]- 


W  '  Ef 

by  choosing  a  =  {  ,  and  we  minimize 


'hk-i+l^gJ  ‘  Ef 


rax 


(2)  p  e  VufiP'  u  til 


F  Xr . ,  -  a  -  ur. 


[i] 


by  choosinp  a  =  [h.O^)  +  hk-i+i^n)l/2  '  Ef- 


min 

ae ») 


as.  ov.m  -  •  -  «fii> 


=  min 
a  e  ( ■  oo) 


V  € 

<W  -  Ef>  - a 

f 


=  0  at  a  *  < 


W  -  Ef 


^k-i+1  ^n^ 


min 


max 


e  (-o“,«o)  u  e  ^(Wjij) 


H  X  r  .  ■»  “  cl  “  pr  .  -I 

F  [l]  [l] 


min  max  ( |h± (f?n)  -  Ef  -  a|,  |hj^_i+1  (Kn)  -  Ff  -  a  |) 

aef-oo,®) 


h.  (p  )  -  hJ  .  .  (p  )  h .  (p  )  +  h.'  .  .  (p  ) 

1  ^n  k-i+lv  n  .  ivrn'  k-i+1  nJ  _ 

- at  a  * - E 

O  ^  J 


c  _>  d)  min  max  ( |c-a  | ,  |  d  -  a  I)  =  (c-d)/2,  as  illustrated 
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It  is  of  practical  interest  to  know  how  any  statistical  procedure 
performs  when  the  (distributional  and  other)  assumptions  under  which  it 
was  derived  are  not  met.  We  then  say  that  (for  deviations  of  a  specified 
sort)  the  procedure  is  'robust”  or  ’not  robust,”  according  to  whether 
the  goal(s)  of  the  procedure  are  or  are  not  met  'well"  under  the 
deviations . 

The  question  of  how  our  procedure  for  estimating  (1  i  k) 

performs  when  specific  distributional  assumptions  are  used  to  set  n, 
but  do  not  hold,  is  answered  in  part  by  our  treatment  of  the  estimation 
problem  for  a  location  parameter  family  in  this  section.  (The  question 
of  robustness  of  Rule  (1.3.2)  is  not  our  concern  here;  for  some  results 
on  this  see  Dudewicz  (1968).) 

The  robustness  interpretation  of  these  results  is  large-sample. 
Small-sample  robustness  can  be  studied  numerically  for  the  f(*)'s 
important  in  any  particular  problem,  utilizing  n.  If  one  is  considering 
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a  location  parameter  family  other  than  the  normal,  results  related  to 
robustness  can  be  used  to  help  design  "good"  procedures,  and  to  help 
compute  the  lose  that  would  result  from  usinp  sample  means  instead  of 
the  appropriate  sufficient  statistic.  If  this  loss  (measured  perhaps 
in  increments  in  n)  were  small  enough,  one  might  wish  to  use  sample 
means  since  they  might  be  more  robust.  (In  any  particular  case  this 
could  be  checked  numerically.) 

Examples  of  location  parameter  families  where  Assumption  (2.1.2) 
holds  but  X^j  is  not  an  asymptotically  unbiased  estimator  of 
(1  <  i  _<  k)  are  presumed  to  exist.  The  case  of  Cauchy  populations 
(excluded  by  (2.1.2))  may  yield  some  insight.  Mere,  (y  |f c)  is 
independent  of  n  (by  a  property  of  means  of  independent  observations 
from  f£) .  (If  Cauchy  populations  were  beinp  dealt  with,  '’ule  (1.3.2) 
would  not  be  used.  See  Dudewicz  (1966),  op.  3°-45.) 

The  relationship  between  the  uniform  intenrability  condition  of 
Theorem  (2.1.26)  and  the  condition  of  Theorem  (2.1.28)  (each  of  which 
is  sufficient)  is  of  interest,  "e  first  clarify  the  role  of  i  (1  _<  i  _<  k) 
in  Theorem  (2.1.26) . 


THEOREM:  Bix  i  (1  <_  i  <^k).  If  |y  |  is  uniformly  inteprable 
(2.1.34)  in  (y  | f ) ,  then  it  is  uniformly  inteprable  in  f (y  | f ) ] 1  and 

1-[1-Cn(y|f)]k'i*1. 

Proof:  For  -<*>  <  a  <  b  <  «•», 


"  b  w 

^|y|dyl(Gn(y|f)]1)  -  i/|y|tGn(y|f)]1'‘dyGn(y|f)  <  i4|y|<yyy|f). 

and  (for  j  _>  1) 

dy{l-[l-Gn(y|f)]j)  -  ♦j[l-Gn(y|f)]j_1dyCn(y|f)  <  jdyryy|f). 
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THEOREM:  E|z,-y|-K)  iff  |z,|  is  uniformly  inteprable  (i.e., 
(2.1.35)  1 

) y  |  is  uniformly  inteprable  in  Gn(y|f)). 

Proof:  Since  E|Zj|<®  (because  EZj  =  y  exists)  and  since  2^  converges 

stochastically  to  y,  the  result  follows  from  the  L^-convergence  theorem 

(see,  e.g.,  Lofeve  (1963),  p.  163,  c.). 


CHAPTER  2.  POINT  ESTIMATION:  BIAS 


2.2.  T'E  NORMAL  CASE 


In  this  section  we  consider  set-up  (1.3.1),  for  which  Rule  (1.3.2) 
was  originally  suggested.  The  fom  of  the  location  parameter  family 
results  of  Section  2.1  is  shown,  and  further  results  are  provided  for 
normal  populations. 

Denote  ( 1  / a)  4> ( y/ a)  by  (Jj^fy) .  Then  the  quantities  defined  in 
Section  2.1  for  a  location  parameter  family  are  (for  i  =  l,...,k)  as 
follows  in  the  case  of  normality. 

f(x-y. )  =  (l/o)*((x-y.)/o)  3  (x-y.); 

1  10  1 


%  =  -^Vy)dy  s  °; 

0 

Gn(yUc)  -  P[X.-W.  <  y) 


o/vfT  o/  Jn 


o/fij 


(2.2.1) 


g  (y  U  )  -  —  — 
n  0  o /Jn 


>/■£ 


h^(gn)  =  E (max  of  £  r.v. ' 


*  Wy); 

s  with  fr.f.  pn(y|»a)] 


=  F[nax  of  £  N(9,a2/n)  r.v. 's] 

=  (o/*^n)E[max  of  £  \'(0,1)  r.v.'s]  =  ( a/ v^T)h ^( d>)  5 
h£(pn}  =  'h£(pn)  =  “Co/^T)h£C«)  *Y  lemma  (2.1.13). 

Note  that  in  the  normal  case,  since  h  (g  )  =  -h'(g  )  =  (o/*^)h  (<}>) 

jc  n  x  n  jC 

(£  =  1,2,...),  only  h ^ ( <t>)  need  be  tabulated.  (h^(4s)  >  0  for  £  _>  2  since 

“  £_1 
/x4>(x)dx  =  0  and  the  positive  weighting  function  [4>(x)]  assigns 

greater  weight  to  +x  than  to  -x  fo^  all  x  >  0.)  Tables  of  quantities 

more  general  than  h^(<}>)  have  been  computed  by  (e.p.)  Teichroew  (1956) 

where  h^ft)  =  B(Xjjt),  and  by  Harter  (1961)  where  h ^ ( <j>)  =  F^Xji^). 


Tables  of  (4>)  have  been  computed  by  Tippett  (1925).  He  now  present 
some  values  of  (<J>)  obtained  from  Harter  (1961)  for  £  =  2(1)10(5) 
25(25)50(50)400,  and  from  Tippett  (1925)  for  £  *  500,1000.  (For 
further  references,  see  Kendall  and  Stuart  (1963),  pp.  329,  336.) 


Table  (2.2.2). 

Values  of 

V*) 

£ 

£ 

n 

2 

.56419 

50 

2.24907 

3 

.84628 

100 

2.50759 

4 

1.02938 

150 

2.64925 

5 

1.16296 

200 

2.74604 

6 

1.26721 

250 

2.81918 

7 

1.35218 

300 

2.87777 

8 

1.42360 

350 

2.92651 

9 

1.48501 

400 

2.96818 

10 

1.53875 

500 

3.03670 

15 

1.73591 

1000 

3.24144 

20 

1.86748 

25 

1.96531 

From  Corollary  (2.1.18),  (2,1.31) (2),  and  (2.2.1),  the  following 
theorem  emerges  for  the  normal  case. 

THEOREM:  For  any  i  (1  <_  i  <_  k) , 

-  (o/«^)hk_itl(<)>)  <  E/[i]  1  +  (<V*/n)h.(<J>) 

and 


(2.2.5) 


is  asymptotically  unbiased  (as  n-*»)  as  an  estimator 
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The  following  theorem  shows  that  the  bounds  of  Theorem  (2.2.3)  are 
actually  the  sup  and  inf.  (For  the  location  parameter  case,  the  inf 
for  i  *  1  arid  the  sup  for  i  =  k  were  proven  as  Theorem  (2.1.21).) 
THEOREM:  For  any  i  (1  <  i  <  k), 
inf{  V[i]  :  »  e  n0(ufi])}  =  u{i]  -  (o/^)hk.i+1U) 

(2.2.4) 

and 

suptE^X^j :  u  e  *  (a/^)h.  («*>) . 

Proof :  By  Theorem  (2.1.15),  the  infimum  is  >y  ^  -  (o/>/n)hk  ^+j(<Ji)  and 
the  supremum  is  fUjjj  +  (°/*^)h^(<f')  •  ?'/e  will  now  show  that 


inf{V[i]:  ^  £  Vw[i])}  -  y[i]  '  (a/^hk-i+lC*) 

sup{E^Xj^j.  y  r  Oo^w[i]^  —  w[i]  +  (o/t^Ohj  (<j>) . 


Now,  since  we  are  taking  the  inf  and  sup  over  more  restricted  sets. 


However,  the  following  weak  convergence  holds  as 
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i-1  terms  k-i+1  terms 

H,(x)-*jx)  =  (x)  with  ip(’»i  < . .  yj.j  |. . . ,  ijjjj) . 

Thus,  by  Theorem  (2.1.25),  if  |x |  is  uniformly  inferrable  in  HfJ,  then 
liin_7xd"M(xl  -frdnjx)  =  U[1J  -  („/^)hk_.tl(*), 
where  the  last  equality  uses  Lemma  (2.1.14)  and  (2.2.1).  Since  |x  |  is 
uniformly  inferrable  in  li,  by  Lemma  (2.2.6),  this  nart  of  the  theorem 
is  proven. 

Case  2.  The  supremum.  By  Lemma  (2.1.10)  and  Theorem  (2.1.11), 

E^X^],  with  ,  •  •  •  ‘^[i]  »ff»  •  •  •  >r;)  »  increases  as  M+.  If  we  let 

JM(x)  denote  Fy  (x)  with  y=(pr.  ■> , . . .  ,yr. , ,!", . . .  ,M) ,  the  desired 
!I  A^j  l1!  1 1 J 

sup{E^Xj^j  •  ys  (y  j  >  •  •  •  >  y  r ^  j  »M> . .  .,H)  e  O^(yr^j)}  “  lii1'_^xdJ^(x) . 

M-k» 

However,  the  following  weak  convergence  holds  as  M-*»: 

i  terms  k-i  terms 

JMCXW.(X)  =  Pjj"  .  (x)  with  y®  (y  j  , .  . . , p ^ j  ,+a>, .  , ,  ,+oo)  . 

The  theorem  follows  as  in  Case  1,  now  using  the  fact  that  |x|  is 
uniformly  inferrable  in  J..  by  Lemma  (2.2.7). 


(2.2.5) 


LEMMA:  For  any  p  c  C(yf4l), 
-  o  t 1 J 


,  k  Tj  W...IV  (X) 

OP,  W  «  l[i  I  t  fx  (x)^flt  (x  S(t)  -U. 

X(i]  «-i  j.l  X8(j)  "aO)1 


Proof: 


F7  (x)  =  P(At  least  i  of  X.,...,X,  are  <x] 

X[i! 


Y  P[Exactly  Si  of  X.,...,X,  are  <x] 

)  4 \  i  i: 


=  J.IT7TTTTT  pfx 


<x,...,X  r  »<x  F>  Xfl,  .  . .  >x, . . .  ,X  f.  >>x] 


g£s  1  e(D- ’•••’  B(£)-  B(£+l)  »•••*■  B(k)' 
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k 

-  I 


M 

k! 


I  F7  (x)...F7  Cx)[1-F7  (x)]...[1-F7  (x)]. 


1- i  Kl  0eSk  "6(1) 


6(£) 


6(fc*l) 


60c) 


Thus , 


\.W  ■  I.TI  .  , 

[l]  £=i  8eS. 


M  f, 

■41  l  I  f  V 


Fy  (x)...F7  (x) 

Aa  /I  x  A  - 


(X) 


8(1) 


mL 


j=l  6  0) 


F7  (x) 

eO) 


[1-F?  (x) ] . . .  [1-F-y  (x)]  -  l  %  (x) 

6(**1)  8(k)  j«i+l  Ae(j) 


Fv  (x)...F7  (x)- 

6(1)  X6(£) 


[1-F7  (x)]...[1-F7  (x)] 

B(U1)  0(k) 


.ilt  ! 


r  tH- 1  i  %  w 


i*i  k!  8eS 


}  L  ay 

.  [j-1  80) 


[1-F7  (X)] 

6(j) 


F7  (x)...F7  (x) 

6(1)  *8(1) 


r 


dx. 


F7  (x) 

X6(j) 


(2.2.6) 


LEMMA:  |x|  is  uniformly  integrable  in  H,,(x)  »  fV  (x)  with 
-  -M  X{i] 

i-1  terms  k-i+1  terms 
u  . •  »"M>^m ( . . . . 


Proof :  Let  L  be  positive.  Then,  by  Lemma  (2.2.5), 

°<  /  |x|<mMw  .  /  |x|dFj  (*) 

|x|>L  |x  !_>L  [i] 


k  w  r 

I  I  kf  I  {if  00 

t=i  8eSk  [j*l  |x|>L  p(j) 


F7  (x)...F7  (x) 

1  '  *6(1)  *B(t) 


F7  (x) 

X8(j) 


dxS 


Fix  any  e>0.  We  will  now  show  that  there  is  an  L  ■  L(e)  such  that  the 

upper  bound  on  /  |x|dHlt(x)  is  <e  regardless  of  the  value  of  M.  By 
|x|>L  M 
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Definition  (2.1.24)  ,  this  will  prove  |x|  is  uniformly  integrable  in 
H.|(x). 

Since  £  *  i,i+l, . . . ,k,  and  since  i-1  populations  have  means  -M 
while  k-i+1  have  means  for  any  fixed  £  and  8  at  least  one  of 

Xg(l)  *  •*  •  *Xg(£)  is  associated  with  a  population  with  mean 

Let  us  consider  the  terms  which  are  summed  in  the  upper  bound  on 
f  |x|dH..(x),  a  typical  one  of  which  is 

Mil 


M 


Fy  (x)...Ft  (x) 

''a  /  <  \ 


T(l.B.j)  =  ifj-  |x|fy  (x)- 
kI |x|>L  X6(j) 


BCD 


XB(j) 


W) 


-dx. 


Case  1.  X0,...  comes  from  a  population  with  mean  y,.,.  Then 
-  BlJJ  lij 

T(£,8,j)  <■*£!-  f  |x|f7  (x)dx 
|x|>L  X8(j) 


and,  since  X^^  is  N(y^,o2/n),  it  is  clear  that  for  L  >_ 
we  have  T(i.B.j)  < 


Case  2.  X„,.^  comes  from  a  population  with  mean  -M,  Then  one  of 
-  pUJ 

X0 Mx.**.,X  .  .  (but  not  Xfl,.J  comes  from  a  population  with  mean  lir.o 
oUJ  ai£j  Pljj  lij 

call  it  X0  .  Then 

P 

0 

fk]  fk] 

T(£,B,j)  i  -vf  /  |xjfy  (x)dx  ♦  J  |x|fy  (x)Ftr  (x)dx. 

x>L  x8(j)  k *X<-L  B(j)  \ 

Since  XQ,.,  is  H(-M,oz/n),  it  is  clear  that  for  L  >  L2C^»B»j  »e)  the 
BUJ  ~ 

1  e 

first  term  is  <  —  ^'.'iVijkTk  uniformly  in  M. 

Now,  since  Xg  is  N(u^j,a2/n),  for  x  <  “ I u [i]  I  tfiat 
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L  >_  L4(£,B,j,y  =  max(L2,L3)  we  have  T(£,6,j)  < 

uniformly  in  fl. 

Using  Case  l  and  Case  2,  since  the  bound  on  /  | x | dl-^Cx) 

|x|>L 

involves  <  (k-i+l)k!k  terms,  we  have  (uniformly  in  M)  /  |x|dH„(x)<e . 

|x|>L 


LEMMA :  |x|  is  uniformly  integrable  in  J^(x) 

(2.2.7) 

i  terms  k-i  terms 

F  3  (u  r  ^  j  >  •  •  •  »P  ^ j  > M )  *  • « ,M)  • 


F-y  (x)  with 

X[i] 


Proof :  Let  L  be  positive.  Now, 

0  <  /  jxjdJM(x)  «  /  |x|dF7  (x). 

"Nil  M  |x|»L  X[i] 

Fix  e>0.  By  Definition  (2.1.24),  to  prove  that  |x  |  is  uniformly 

integrable  in  Jf1(x) ,  it  is  sufficient  to  show  that  there  exists  an 

L  »  L(e)  such  that  /  |x|dJM(x)<e  for  all  M. 

I*l>i.  1  " 

For  M  >  [i] I  *  Theorem  (2.1.11), 

i  times  k-i^times 
JM(x)  3  Fy  (x)  With  -Cu— . M) 

i-l> times  _k-i+l  times 
—  ^x"  .  with  y  =  ( >  •  •  • ,  ~M,y  ^  j  >  •  •  •  >  y  ^  j  ) 

=  Hf.jW. 


Define  two  d.f .  's 

1  if  x  >  -L 


if  x  >  -L 


F(x) 


G(x)  =  < 


I  JM(x)  if  x  <  -L 


|H^(x)  if  x  <  -L. 


Then  by  Lemma  (2.1.10), 
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2xdF(x)  >  _7:cdG  (x) 


-L  -L 

jTxdJM(x)-L(l-JM(-L))  >  _/xdHM(x)  -  Kl-Hj^C-L)) 

-L  -L 

0  1  _£KdJ. M(x)  >  ./xdHfJ(x)  ♦  L{Hm(-L)-Jm(-L)}. 

-L 

Now,  since  HM(-L)  _>  and  since  £xdN  (x)->0  uniformly  in  M  by 


Lemma  (2.2.6),  we  find  that 


-L  -L 

0  >  /xdJ,.(x)  >  /xdH.,(x)-»0  uniformly  in  M. 

—  -oo  M  —  -oo  M 

Thus,  there  is  (for  any  fixed  p^j)  an  Ljfe)  such  that  for  L  >  Lj(e) 

-L 

we  have  /  |x  |dJ..(x)  <e/2  uniformly  in  M. 

Take  L  >  Lj(e).  By  Theorem  (2.2.3)  and  Theorem  (2.2.4),  we  have 

y  ^  j  +  (  o/  *6jh.  ( 40  sup  ^  j  :  lJ=(w^j,...,)J^j,M,...,M)  c  ^(pj-^j)} 

oc  -L  L  oo 

=  lim  /x dJ.,(x)  *  lim  {  /xdJ,,(x)  +  /xdJM(x)  +  /xdJ„,(x) } 
M-*»  11  m-k»  “  M  -L  ri  L  H 


L  oo 

>  -e/2  +  lim  fx dJ..(x)  +  lim  /xdJ,.(x) 
!■»*»  -L  H  M-*»  L  m 


The  last  step  follows  from  the  Helly-Bray  Lemma  (as  in  (2.1.27)). 
Since  (as  shown  in  Theorem  (2.2.4)) 

.ZxdJJx)  =  +  (o//n)h.($), 

L 

for  L  >  L2(e)  we  have  /xdJ  (x)  within  e/2  of  prl  ♦  (o/^n)h  .(<}») . 

-L  00  l1'  1 

Thus,  if  L  >  maxfL^^)  then 

p^j  +  (a/v/rT)hi (4,)  i  "f  +  u  [ij  +  (a/ /n)hi(4>)  +  lim  7xdJM(x) 
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e  >  lir  7xdJ  (x) . 

~  M-kd 

Thus,  there  is  an  L  =  L.(e)  such  that  f  |x|dJM(x)<e  regardless  of 

IxIiL  A 

the  value  of  M. 

Among  the  results  of  Section  2.1  for  a  location  parameter  family 
which  ergo  hold  for  the  normal  family  of  the  present  section,  the  linear 
corrections  for  (e.g.)  minimax |bias  |  at  equation  (2.1.32)ff  are  worthy 
of  special  note.  We  may  then  (in  the  normal  case)  readily  determine 
the  sample  size  n  needed  to  satisfy  several  criteria  (ranking  and 
selection,  estimation,  or  both).  (1)  Set  n  as  dictated  by  the  ranking 
and  selection  use  of  Rule  (1.3.2),  say  n^.  (2)  Set  n  to  make  certain 

minimaxjbias | 's  suitably  "small,"  say  n.,.  (3)  Set  n=max(nj ,n2) . 


Table  (2.2.2)  of  values  of  h^(<j>)  indicates  that  for  k  in  the 
range  in  which  Rule  (1.3.2)  would  usually  be  used  (k  <_  10)  the  factor 
h  (<J>)  in  the  bias  is  not  seriously  detrimental,  being  only  1.5  for 

X* 

£,  *  10.  Even  if  I  were  of  the  size  associated  with  large  screening 

experiments,  the  factor  («f> )  would  still  be  only  3.0  for  i  =  500. 

As  an  example,  if  one  were  setting  n  large  enough  to  make  the  minimax 

| bias |  in  -  a,  as  an  estimator  ofy^,  ^  e  (e  >0),  he  would  find 

approximately  that  if  nQ  sufficed  for  k  =  2,  4nQ  would  suffice  for 

k  *  5:  and  that  if  n  sufficed  for  k  *  9,  4n  would  suffice  for  k  =  500, 

o  o 

since  by  Theorem  (2.1.33)  the  minimax  (bias  |  is 


=  2  (o/*/n)hk(<|>) 


Note  that  if  there  are  restrictions  on  the  y.  (i  *  1, . . .  ,k)  in  a 
practical  case,  then  the  inf  and  sup  of  Theorem  (2.2.4)  can  be  improved. 


j 


1 
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For  example,  if  A  <_y^  B  (i  =  then  "A"  will  replace 

and  '  B"  will  replace  "+<*>'•  in  that  work.  (A  common  case  is  A  *  0, 

B  =  ■*».)  Such  a  process  will  result  in  a  smaller  n^  being  needed 
for  estimation  as  in  the  previous  paragraph. 

If  the  sup  and  inf  were  desired  over  a  more  restricted  set  than 
y  e  fio^[i]^’  say^  e  [i]^’  *^at  SUP  wou*^  also  attained 

by  raising  (lowering)  the  components  of  u  to  the  highest  (lowest) 
possible  values.  Moting  that  this  is  somewhat  analogous  to  the  set 
over  which  a  Probability  Requirement  is  made  in  the  "indifference 
zone"  formulation  of  ranking  and  selection  problems,  one  might  at 
first  think  we  would  be  interested  in  the  sup  (inf)  over  y  e  fi^(y^,). 
However,  since  our  aim  is  good  estimation  of  y  ^  regardless  of  y, 
the  set  used  above  (y  e  fiQ(y  ^j))  will  usually  be  the  proper  one. 

(For  special  uses  of  the  estimate  of  y^j  one  may  only  "care"  when, 
for  some  6,  y  e  Qg(y^).) 


CHAPTER  3.  POINT  ESTIMATION:  STPOMG  CONSISTENCY 


3.1.  STRONG  (W.P.  1)  CONSISTENCY  OF  A  NATURAL  ESTIMATOR  OF  (l<i<k) 

FOR  A  LOCATION  PARAI^ETER  FAMILY 

Consider  X^j  as  an  estimator  of  y  ^  (1  <  i  <_  k)  when  Set-up 

(2.1.1)  and  Assumption  (2.1.2)  hold,  i.e.,  when  observations  from  pop¬ 
ulation  have  fr.f.  f(x-6.),  XeR,  i  =  l,...,k, and  the  mean  of  f  exists. 
If  Z  is  a  constant  (say  0)  with  probability  one  (w.p.  1),  a  sequence  of 
estimators  { Z  ;  n  ^  1}  is  said  to  be:  strongly  consistent  (for  0)  if  Z 
converges  to  0  w.p.  1;  consistent  (for  0)  if  Z  converges  to  0  in  prob¬ 
ability.  Since  convergence  w.p.  1  implies  convergence  in  probability, 
strong  consistency  implies  consistency. 

LEMMA:  Let  Tj (n) , . . .  ,T^(n)  (n  1)  be  r.v.'s  which  converge 
w.p.  1  to  r.v.'s  Tj,...,^  (respectively).  Suppose  that 

g(t.,...,t.)  is  a  continuous  function  of  k  real  variables. 

(3.1.1)  1  K 
Then 

g(Tj (n),...,Tk(n)) 
converges  w.p.  1  to  g(Tj, . . .  ,T^) . 

Proof :  Suppose  that  all  r.v.'s  involved  are  defined  on  a  probability 
space  (n.B.P).  Then  by  a  characterization  of  convergence  w.p.  1 
(see,  e.g.,  Parzen  (1960),  p.  415),  it  suffices  to  prove  that  for  every 
c  >  0,  6  >  0  there  exists  an  integer  Nq  >  0  such  that 

P[sup  | g (T.  (n) , . . . , T,  (n) ) -g (T. , . . .  ,T. )  |  >  e)  <6. 

n>N 
-  o 

However,  by  the  continuity  of  g (•>•••>')  and  the  convergence  of  T.(n)  to 
Tj  w.p.  1  (1  i  i  Ik),  this  is  clear. 
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THEOREM:  Xr>1  is  strongly  consistent  as  an  estimator  of 
(3.1.2)  L1J 

O  1  i  I  *)• 

Proof:  Since  7xf(x)dx  is  assumed  to  be  a  finite  number,  it  follows 

-  ■■■  ■■■'  "■  —  oo 

by  Kolmogorov's  Strong  Law  of  Large  Numbers  (see,  e.g.,  Lo^ve’ (1963) , 
p.  239)  that  Xj ,X^  converge  w.p.  1  to  y  ^ ,...  ,u^  (respectively). 

Thus  by  Lemma  (3.1.1)  X^j  converges  w.p.  1  to  y^  (i  =  l,...,k). 


The  stronger  theorem,  that  g(X^j)  converges  w.p.  1  to  g(y^)  for 
any  continuous  real-valued  function  g(  •)  (1  <_  i  <_  k)  is  obvious.  It 
can  be  used  as  follows:  "(X^j)  may  be  use^  ho  yield  an  estimate  of 
g(Ujk])»  where  g(0  is  a  continuous  function  such  that  if  we  knew  the 
mean  of  the  selected  population  to  be  y  ,  then  we  would  know  the 


expected  worch  to  us  (e.g.,  in  dollars)  of  the  selected  population  to 
be  g(y).  Other  applications  might  occur  for  a  Bayesian  taking y  ^  to 
be  a  r.v.  (1  <  i  <  k)  . 

Note  that  strong  consistency  of  as  an  estimator  of  y^j 
implies  strong  consistency  of  X^j  +_  a^  where  lim  a^=  0  (i  *  l,...,k). 


(This,  of  course,  was  also  the  case  for  asymptotic  unbiasedness.) 


CIIAPTFR  4.  POINT  ESTIMATION:  SQUARED  F.PROP 


4.1.  SQUARED  ERROR  OF  A  NATURAL  ESTI'IATOR  OF  (l<i<k) 

FOR.  A  LOCATION  PARAMETER  FAMILY 


In  this  section  we  consider  the  squared  error  of  as  an  estima¬ 
tor  of  (1  <_  i  <  k)  when  Set-up  (2.1.1)  and  Assumption  (2.1.2)  hold, 
i.e.,  when  observations  from  population  have  fr.f.  f(x-0i),  x  e  R, 
i  *  l,...,k,  and  the  mean  of  f  exists.  The  expectation  of  this  ouantity, 
i.e. 


(4 . 1 .  i)  Ey(X[i]'u[i])2* 

will  be  of  special  interest. 

LEMMA:  If  F(-)  and  G(-)  are  d.f.'s  with  F(x)  <  G(x)  (x  e  R) , 
then  for  i^(x)  any  monotone  non-decreasing  function  of  x  we 
have 


(4.1.2) 


.Z'Kx)dG(x)  1  „jKx)dF(x), 

with  the  inequality  reversed  if  i^(x)  is  monotone  non¬ 
increasing. 


This  lemma,  which  is  a  generalization  of  Lemma  (2.1.10),  has  been  essen¬ 
tially  stated  by  Alam  (1967),  p.  283,  who  refers  to  Lehmann  (1955)  for  the 
proof.  That  reference  is  concerned  with  more  general  questions  (which 
makes  it  difficult  to  extract  the  needed  proof) .  A  simple  proof  (for 
the  strictly  monotone  \j>(*)  case)  is  possible  usinp  the  inverse  func¬ 
tion.  We  omit  this  since  Mahamunulu  (3967),  p.  1082,  has  recently  pub¬ 
lished  a  reference  on  this  result. 
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(4.1.3) 


DEFINITION: 


For  our  location  parameter  family,  let 

1-1  ^erms  k-i+1  terms 
(x)  withy  -  C-00, ^j) 


J  (x)  =  Ftt  (x)  with  y 
X[i] 


i  terms 


k-i  terms 
+«0  . 


Although  H  (•)  and  Jro(.)  depend  on  i  (1  <_  i  <_  k),  this  dependence  will 
be  suppressed.  (We  used  this  notation  for  the  normal  case  in  Theorem 
(2.2.4).) 

LE'MA:  For  any  monotone  non-decreasing  function  of  x  i>(x)  and 
y  e  %(yti]), 

M  1  4)  i^w^w  1  Jfylx)^  M  I  I^McUjx) 

[i] 

(i  =  1, . .  .,k),  with  both  inequalities  reversed  if  ij/(x)  is  mono¬ 
tone  non- increasing. 

Proof :  This  follows  from  Theorem  (2.1.11)  and  Lemma  (4.1.2). 


THEOPEM:  For  any  i  (I  <_  i  <_  k)  and  any  y  e  fi0(F[jj)» 


Vfi 


[i] 


(4.1.5)  u [i] 


/  (x'u[i])2dH00M  +  . I  (X'F  [i])2dJoo(x)  I  Ey^X[i]"w[i]-) 


U  [i] 

<  _J  (x-uCi])2dH Jx)%^Mx-y[i])2djjx). 


Proof:  Define 


41 


*j(x) 


f if  X‘M  [i]  >  0 


..  f 


^2(X)  “  )  , 


if  x-m^j  ^  0 


if  x-u(i]  >  0 


[i]’ 

Then  by  Lemma  (4.1.4),  since  tfrj(x)  is  monotone  non-decreasing  in  x 
and  ^(x)  is  monotone  non-increasing  in  x, 


1 .1. (XH1  [i ] ) 2dMx)  i ..  .7  (*->*  m )  2dF7.  .Ml.].  Mu  r .  i )  2dJ0O(x) , 


[i] 


m/  u,v  (.-'V  _  1  Ti  1. 

[i]  ^ [i]  [lj 


Wti] 


[i] 


U[i) 


L  ^XHJ  [i])2dH®(x^  1  !  (X'y[i])2dFXj .-jCX^  -  J  Cx-V [i] )2<iJ«,Cx)  0, 


from  which  the  theorem  follows  easily. 


Note  that  since  (for  any  r.v.  Z)  EZ2-(EZ)2  =  Var(Z)  and  since 
Corollary  (2.1.13)  gives  us  bounds  on  Theorem  (4.1.5)  can 

be  used  to  obtain  bounds  on 


Var  Xr., 

u  [i] 


VaVX(i]1J 


[i] 


E  (Xr.,-u  ,..,)' 
w  [i]  [i]' 


-  (E  Xr.,-pr.,)2. 
VJ  [l]  [l]J 


CHAPTER  4.  PCINT  ESTIMATION:  SQUARED  ERROR 


4.2.  THE  NOPJ1AL  CASE 


In  this  section  we  first  find  the  form  of  the  results  of  Section 
4.1  in  the  case  of  normality.  Under  normality, 

II^Cx)  »  P[Miniraum  of  k-i+1  N(y^^,o2/n)  r.v.'s  is  <x] 


x-yf . 


«  1-  1-* 


=  P  Min  of  k-i+1  N(0,1)  r.v.'s  is 


k-i+1 


J^Cx)  =  P[Maximum  of  i  N(yj^,o^n)  r.v.'s  is  <x] 


P  Max  of  i  N(0,1)  r.v.'s  is  <- 


a/J n  . 


Thus, 


®  00  if  jj.y 


(o2/n)/x2d{-[l-4(x)]k"1+1}  -  (o2/n)  /x2d{ f4>Cx)]K'1+1} ; 


k-i+li 


J  (x-Ur.^dJ  (x)  =  /  (x-wnl)2d'  4> - >  •  (o2/n)  /x2dj  [e(x)] 1 V; 


[i]  um  [  r  fx-nri1]ik-i+i 

r  _  r  , _  ■» 2 jJ i  li  *  AiJ 


(o2/n)  /x2t'|-[l-$(x)]I<"i+1j  .  (o2/n)/x2djtO(x)]k'i+1|; 
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“j](*'“<i))2<u*00  *  M 

Thus,  by  specializing  Theorem  (4.1.5)  to  the  case  of  normality  and  using 
the  above  results,  we  obtain  the  followinp  theorem. 

THEOREM:  For  any  i  (1  <_  i  <_  k)  and  any  y  e 


(4.2.1) 


(o2/n)  Jx2d-  4>(x)  k"i+1J  +  (o2/n)  Jx2dj  4>(x)j1j  <_  Ey  (X^j-y 


1  (o2/n)  /x2d|  4>(x)  + 

o  ‘  L  J  J 


In  the  case  of  normality,  it  is  possible  to  further  bound  the 
supremum  and  infimum,  thus  obtaining  an  interval  in  which  each  must  lie, 
THEOREM:  For  any  i  (1  <  i  £k),  taxing  the  inf  and  sup  over 

y  e  V^i])’ 


inf  Ey(X[i]’y[i])2 


(4.2.2) 


<  min 


i|(a2/n)  Jx2djj$(x)jk"1+1 (o2/n)  Jx2dj|$(x) 
sup  HlfXm.[i])2 


>  max 


(o2/n)  Jx2dj|$(x)  k‘i+1j  (o2/n)  /x2d| jjs (x) 


Proof :  Since  (see  Theorem  (2.2.4))  Hj,(x)  and  JM(x)  converge  weakly  to 
H^fx)  and  J^x)  (respectively),  by  Theorem  (2.1.25)  it  follows  that,  if 
x2  is  uniformly  integrable  in  H,j  and  JM,  then 


00  oo  00 

lim  j(x2dH,,(x)  *  Jx^H^x)  *  (a2/n) 

f)-H»  -00  -  00  -  00 


00  00  00 

lim  /x2 dJfJ(x)  =  /x2dJoo(x)  =  (o2/n)  Jx2d{f4>(x)]i}. 

-00  -  00  -  00 

In  this  case  it  must  be  the  case  that  the  inf  (sup)  is  less  (greater) 

than  or  equal  to  each  of  these  quantities. 

The  fact  that  x2  is  uniformly  inte^rable  in  H^follows  from  a  modifi 

cation  of  the  proof  of  Lemma  (2.2.6). 

The  fact  that  x2  is  uniformly  inteprable  in  J  requires  major  modifi 

M 

cation  of  the  proof  of  Lemma  (2.2.7),  as  will  now  he  noted.  Using 

Lemma  (4.1.4)  with  the  non-increasinp  function 

jx2,  x  £  -L 
iHx)  =  < 

[0  ,  X  >  -L 

ead  of  Lemma  (2.1.10))  we  find 


/^(x)dG(x)  >  /^(x)dF(x) 

-GO  -00 

-L  -L 

/x:dUM(x)  ♦  L2(1-Um(-L))  >  /x2 dJM(x)  +  L2(1-JM(-L)) 

_  00  —  00 

-L  -l 

/x2dHf.(x)  >  /x2dJ.](x)  ♦  L2{Hm(-L)-J„(-L)  ). 

-00  -00 

■*  L 

Now,  since  II,,(-L)  >  Jf)(-L)  and  since  /x2 d"rj(x)-*0  uniformly  in  M,  we 

-  oo 

find  that 

-L  -L 

0  <_  /x2dJ.,(x)  <_  /x2dKM(x)-*0  uniformly  in  '1. 

-00  -00 

Thus,  there  is  (for  any  fixed  an  Lj(e)  such  that  for  L  >  L^(e) 

-L 

we  have  Jx2dJM(x)  <  e/2  uniformly  in  M. 
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By  Theorem  (2.1.11),  J^(x)  >  J^fx) .  If  we  define 


F(x) 

G(x) 


JM(x),  x  >  L 
0  ,  x  <  L 

J»00.  x  1  L 

0  ,  x  <  L 


<H*) 

then  by  Lemma  (4.1.2), 


x2  ,  x  ^  L 
0  ,  x  <  L 


iMx)  dF(x)  i  /^(x)dG(x) 


00  00 

/x2dJ  (x)  ♦  L2J  (L)  <  Jx2dJoo(x)  +  L2J  (L) 

L  L 


0  I  fx2dJ.,(x)  i  L2{J  (L)-J,.(L)  }  +  /x2dJoo(x)  <  fx2dJJx). 
L  L  L 


Now  since  /x2d.Joo(x)  exists,  for  L  >  L2 Ce)  we  have  /x2dJ.,(x)  £  e/2 


uniformly  in  ff.  The  result  then  follows  as  in  Lemma  (2.2.7). 


We  now  find  the  min  and  max  needed  in  Theorem  (4.2.2).  This  will 

allow  us  to  specify  intervals  in  which  the  inf  and  sup  must  lie,  and  to 

study  the  lengths  of  these  intervals. 

LEMMA:  Let  Zj,...,Z  be  independent  r.v.'s,  each  with  d.f.  F 

such  that  Ffi-)  *  :(-z)  =  1  for  all  z  (e.p.,  this  occurs  if 

F  has  a  fr.f.  which  is  symmetric  about  0).  Let  f>  (z)  be  the 
(4.2.3)  n 

max  Z.  .  Let  h(u)  be  any  non-decreasing  function 
l<i<n  1 


d.f.  of 
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of  u>0  such  that  h(0)>-».  Then  Jh(u)dGn(u)  is  non-decreasing 


in  n. 


Proof :  For  u  >_  0,  Gn+^(u)  £  G^(u)  (n  =  1,2,...)  since 


Gn(u)  =  P 


["  I  max  X .  I  <_  u  ]  -  P  -u  <_  max  X .  <_  uj 

|_|l^i<n  1|  J  _  l<i<n  1  J 

F  max  X.  <  u]  -  Pf  max  X.  <  -  ul  =  Fn(u)-[1-F(u)]n 
|_l<i<n  j  |l<i<n  J 


implies  that 


Gn(u)  -  G^l(u)  = 


o  if  n  *  1 

l  F(u)[l-F(u)][Fn‘1(u)-Fn"1(-u-)]  if  n  >  Oj 


>  0. 


Hence  the  desired  result  follows  from  Lemma  (4.1.2) 


COROLLARY;1  Jx2d{  [ <t> (x)  ]n)  =  1  for  n  *  1,2  and  is  a  strictly 

(4.2.4) 

increasing  function  of  n  thereafter. 

Proof :  Choosing  h(x)  =  x2  and  F  =  $,  by  Lemma  (4.2.3) 

oo  oo  oo 

/x2dG^(x)  =  Jx2d{ [^(x)]n)  -  /x2d{  [l-$(x) ]n} 

0  0  0 

00  3  00 
=  Jx2d{  [4>(x)  ]n}  +  /x2d{ Tf (x) Tn)  *  /x2d{ T ^ (x) ] n } 

0  -oo  -  oo 

is  non-decreasing  in  n. 


THnOPF.il:  For  any  i  (1  <_  i  <_  k) ,  inffF^  (X^-p  j^j)2  : 

u  c  Q  fci  ,• i))  is  in  the  closed  interval 
o  [  1  J 
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|(o2/n)  Jx2d|[$(x)]k'1+1j  +  (o2/n) 


/x2d|[4>(x)  ] i|. 


(a2/n)  /x2d|[4>(x)]k'i+1|.  if  i  1 


^(o2/n)/x2dj[$(x)]k'1+1 j  ♦  (o2/n)  /x2d|[«(x]j1j  , 


(4.2.5) 


(a2/n)  /x?d|[4>(x))1| 
e  fiQGj  [i]^|  *s  *n 


...  .  .  V+1 

if  i  <  — 


and  sup|n^(Xj^-y  jjj)2:  U  e  ftQGi  [i]^}  *s  *n  t*ie  c^osec*  interval 


|(o2/n)  Jx2dj[$(x)]3 j, 

(o2/n)  /x2d|f$(x)]k‘i+  )  ♦  (a2/n) /x2d|[4>(x)]i|J  if  i  >_ 
^(o2/n)  ]x2d|[4>(x)]k'i+1j, 

(a2/n)  Jx2d-  [«(x))k'i+1j  ♦  (o2/n)  /x2d|f4>(x)]1|  if  i  < 
o  1  o  1 

Proof :  See  Theorem  (4.2.1)  for  the  lower  (upper)  end  points  on  the  inf 
(sup),  and  Theorem  (4.2.2)  with  Corollary  (4.2.4)  for  the  other  end 
points. 
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(4.2.6) 


COROLLARY :  The  inf  and  sup  of  Theorem  (4.2.5)  each  lie  in  an 
interval  of  length 

(o2/n)  (7x2d(  [4>(x)]k’i+1}  -  _£x2d{f4>(x)]*})  if  i 

(o2/n)  (7x2d{  [ 4>(x) ] 1 }  -  _?x2d{f4>(x)]k"1  +  1})  if  i  < 

By  Corollary  (4.2.4),  the  intervals  of  these  lengths  for  the 

k+ 1 

inf  and  sun  fail  co  be  disjoint  iff  (i  =  — i » or  (i,k-i+l)  is  a 
permutation  of  (1,2)).  In  that  case  they  have  exactly  one 
common  point. 


CHAPTER  5.  POINT  ESTIMATION:  MAXIMUM  LIKELIHOOD  (ML) 

AMD  RELATED  ESTIMATORS 
5.1.  MLE ' s  FOP  ] 

Consider  first  maximum  likelihood  estimation  of  yj,...,yk;  i.e., 
we  seek  the  maximum  likelihood  estimators  (MLE's),  those  functions 
Ul,...,uk  (if  such  exist)  such  that  the  density  of  the  observed 
statistics  (whatever  they  may  be)  is  maximized  by  settin" 

“1  *  v  -uk  =  V 

Our  observed  statistics  under  Pule  (1.3.2)  are  (i  =  l,...,k; 
j  *  l,...,n),  but  since  X. , . . .  ,Xj.  are  sufficient  statistics  we  may  tahe 
them  as  fundamental.  Then 

r-  V  fXl"Ml) 

(5.1.1)  fv-  7(x. . x,.)«(W*  —  •••♦  - - 

Xl’"*’Xk  1  [a/ /H  a/JR 

and  (if  y^  t  y_.;  i  /  j  ]  i,j  *  l,...,k)  the  MLE's  of  y|,...,yk  base^  on 
Xj . Xk  exist  and  are  uniquely 

(5.1.2)  ux  =  Xj,  ...  ,uv  =  Xk. 

(Tiie  restriction  to  MLE's  based  on  Xj,...,Xk  is  a  consequence  of  the 
peneral  result  that  MLE's  are  functions  only  of  sufficient  statistics 
for  a  problem;  see,  e.n.,  "of"1  and  Craip  (1965),  pp.  245-246.)  The 
problem  of  possible  equalities  amone  y j j j , . . . ,y ^k j  is  discussed  below: 
similar  results  hold  for  the  case  of  equalities  amo”"  yj,...,yk. 

For  the  problem  of  findinp  an  MLF  of  a  1-1  function  u(y  j , . . .  ,u,.) , 
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it  is  well-known  that  (assuminr  the  MLE  of  pj,...,pj.  exists) 
u({!  ,...,p.)  =  u  (say)  furnishes  a  solution,  essentially  because  forcing 
u=u  implies  p^  =  p^,  ...  ,  p  ^  =  p,,.  (See,  e.g.,  Hoge  and  Craig  (1065), 
p.  247.)  If  u(p j , . . .  ,p ^)  is  not  1-1,  i.e.  if  it  is  many-to-one,  points 
other  than  Pj  *  Pj,  ...  ,p^  =  pj,  nay  also  be  implied  by  u  =  u.  In  this 
case  Zehna  (1966)  was  the  first  to  state  explicitly  a  reason  for 
picking  only  the  'right'  noint  p^  =  p ^ ,  ...  ,  p  t.  =  p^  for  attention 
(and  thus  for  calling  u  an  f'LE).  Berk  (1%7)  gives  a  different  justi¬ 
fication  for  calling  u  an  MLi\ 

From  the  above  it  is  clear  that,  based  on  Xj,...,X^, 

(5.1.3)  p^  =  {ith  smallest  of  , . . . ,  }  =  X^j  (i  =  l,...,k) 

is  the  Berk-Zehr.a-flLE  of  Pj^,...,Pp^.  Below  we  discuss  the  problem 
of  MLE-tyne  estimators  of  (p  , . . . ,  Pp,-j )  fro r  another  noint  of  view. 
This  rethod,  Tterated-ML.F ’s ,  is  discussed  in  Section  5.2. 

Elumenlhal  and  Cohen  (l°68a),  (1968b)  (wbo  provided  the  author 
with  preliminaries  of  their  papers)  studied,  for  a  translation  param¬ 
eter  family,  (1)  estimation  o*  t’ e  pair  (pj^.p^)  for  the  sun  of 
squared  errors  as  loss  function  and  (2)  estimation  of  ^  for  a 
squared  error  loss  function . 

Other  work  on  the  case  k  =  2,  in  another  formulation ,  was  done  by 
Katz  (1963),  who  nroposed  to  estimate  (p  f  n  ^  [2  j )  when  one  knows  that 
(e.g.)  Ttj  is  associated  v/ith  p  ^  and  is  associated  with  p  ^  .  This 
work  was  done  for  binomial  nrobabilities  and  also  for  normal  means,  with 
(e.g.)  sum  or  squared  error  losses.  (Die  fact  that  (X^,!^)  *s  not  a 
totally  desirable  estimator  may  b"  seen  intuitively  from  the  fact  that, 
although  _<  b[2]’  *n  fenera^  >  can  occur  with  positive 
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probability.)  In  our  work  one  does  not  know  the  association  of  the  y^ 
with  the  it  (i,j  =  l,...,k);  see  Robertson  and  Maltman  (1968)  for  the 
case  where  one  does. 


Blumenthal  and  Cohen  (1068),  who  utilize  the  MLB  of  y  found 

below,  desired  theiv  estimate  to  he  symmetric  in  X.,X0;  in  order  to 

1  z 


force  this  they  based  their  estimate  on  the  maximal  invariant  X 


[1]’ 


X j- 2 j  *  >,ote,  however,  that  in  order  to  obtain  symmetry  in  X^.X^  (and 
certain  other  invariance  conditions)  in  one's  estimator,  one  need  not 
?,o  to  ’ X  [  2  ]  (at  least  for  the  normal  case;  see  (5.1.3)).  Mote  that 

(although  the  MLL  of  y^j  based  on  is  X^j)  the  °f  w ^ 2)  ^asetl 

on  not‘  Section  5.2  we  give  additional  justification 

for  basing  the  MLB  on  ^m,Xfj»j. 

"e  will  now  consider  the  general  case  in  which  it  is  desired  to 
find  the  MLr's  of  y  ^  , . . .  ,y  ^  based  on  1  ]»•**, V fv ]  *  ^*ie  likelihood 

function  is  given  in  (B.1.1),  and  (due  to  its  symmetry  in  u  j j-j  » •  •  •  »u  rvj ) 
if  tf  , . . .  ,U  is  an  then  so  is  any  permutation  of  it  (so  that  it 
is  not  necessarily  the  case  t’-at  y  ^  ^  <_. . .  <y  ]  )  .  In  order  to  eliminate 
such  undesirable  occurrences,  we  require  a  consistency  condition. 

CONSISTENCY  CRITBRION:  /rone  the  (at  most  k!)  permutation 
(5.1.4)  MJ.R's  which  any  i)  j  j  j , . . .  ,u  which  maximizes  (B.1.1)  pro¬ 
vides,  only  the  one  with  0  jr  j  jl*  •  •  will  he  called  an  MLr. 


From  (B.1.1)  and  the  form  of  $(  ),  it  is  clear  that  we  nay  restrict  our 
search  for  the  maximum  to  y  j.  ^  , . . .  ,u  ^  sue'1  that  <  {y  ^  ^ , . . .  ,y  } 

<  x^..  By  (5.1.4)  we  need  only  consider  t’  c  case  u  r  1 1  <  •  •  rj^  ,  a001  not 
all  k!  (fewer  if  there  are  any  equalities)  or^erin^s.  It  is  well-known 
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(see,  e.g.,  Hancock  (1960),  p.  80)  that  in  such  a  case  the  maximum  must 
occur  at  y  , . . .  ,y  ^  j  such  that 

3fy"  V  (X-  ,  .  .  .  ,  X,  ) 

Am  > "  •  »An  i  1  K 

(S.1.5)  _ - LJ - 0 


3jj 


(i  *  l,...,k); 


[i] 


any  point  u  ^  j , . . .  ,u  jj.  ^  (which  depends  on  the  values  of  Xj,...,x^)  where 

(5.1.5)  holds  is  called  a  critical  point. 

In  takin"  the  derivatives  (5.1.5),  the  results  depend  on  how  many 
of  the  k-1  inequalities  are  equalities.  There  are  thus 


,k-l 


mutually  exclusive  and  exhaustive  cases,  say 


(5.1.6) 


ft  =  o +  •••  +  n.^k-i. 
o  (1)  (2)  ("  ) 


where  the  are  disjoint,  =  n(^)  is  defined  in  (1.3.12),  and  the 

!-  - 1  V  - 1 

ft^  (i  =  2,...,2  )  are  the  other  2  -  1  cases  in  some  order.  Fix 

k-1 

any  i  (2  _<  i  <  2  )  and  suppose  that  some  y*  e  solves  the  system 

(5.1.5)  (i.e.,  is  a  critical  point  when  the  derivatives  are  taken  for 
\i  e  ft^).  easy  to  verify  (usi"P  (B.1.1))  that  y*  is  a 

critical  point  of  system  (5.1.5)  when  derivatives  are  taken  for 
y  e  ft^.  we  thus  have  the 

THEOPT-H:  Any  critical  point  for  our  problem  is  a  solution  of 
system  (5.1.5)  with  derivatives  taken  for  y  e  ft(?0,  provided 


(5.1.7) 


only  that  we  allow  boundary  points  (i.e.,  points 


(2) 


♦  ...  ♦  ft^k-lj)  t0  considered  solutions. 


To  completely  justify  calling  the  boundary  points  included  in  'Tieorem 
(5.1.7)  critical  points,  one  should  show  that  any  such  point  is  a 
solution  or  system  (5.1.5)  when  derivatives  are  taken  for  y  in  its 
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this  is  clear  from  the  proof  of  Theorem  (5.1.7). 

Now  (taking  derivatives  when  u  ^ .  .cy  system  (5.1.5)  is 


(5.1.8)  l  {/ri/or* 
BeS. 


xed)^tn 


l  0 


/ft  J 


*  d> 


XB(k)~y[k] 

xBti>'u  m , 

;  a/ ft 

a/ ft 

(i  =  1, . . . ,k)  , 


or 


^  XB(i)* 


(5.1.9)  i),.| 


BeS, 


X8(i)~u  n] 


. .  4> 


/ft  J  (  a/ft  J 


B(k)'y  fk] 


l  ♦ 

SeS. 


XB(1>'U[1] 


[  a/ ft  j 


. .  $ 


x?(k)~y  [k] 


(i-l,....k) 


l  a/ft  J 


or 


(5.1.10) 


PJjl 

P[i] 


BeS, 


wr 


s(i)~ym 


[  a/ ft  J 


. .  4> 


CB(k)*[k'j 


[  a// n 


^  X8(i)* 


BeS, 


,w)~>‘ni 


\  a/ ft  j 


B(k)~>] 


l 


a/ ft  J 


-(i.j-1*  »k;i<j) 


_  _  X.  +  ...+Xj, 

THEOREM:  (y j!j,...,y pk])  *  (Xj...,x)  with  x  = - 

(5.1.11)  ‘  k 

is  a  critical  point. 

Proof:  It  is  clear  that  this  is  so  from  system  (5.1.9). 


We  will  now  investigate  the  nature  of  this  critical  point.  For 
i ,  j  *  1 , . . . , 1  ■ ,  for  x^.-.o^. 


g2  ^  ^  ^ 

t  r  fy  y-  (x«  *  • . .  »x. ) 

^[i]  [j]  [1] .  [k]  1 
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(5.1.12) 


i  i  j 


j- 


Thus,  for  the  matrix  0  *  (d^)  evaluations  of  (5.1.12)  at  (jc, . . . , x) 
we  find 


(5.1.13) 


«  (k-2)!(^/o)k+* 


jcov(" ,S)k(k-l) 

> 

lk(k-l)var(P)  - 


l 


.  i  t  j 

k(k-l) (o2/n)f 
i  «  j 


and  Theorems  (A. 2.1)  and  (A. 2. 2)  pive  us  the 


(5.1.17) 


THEOREM :  The  nature  of  the  critical  point  (x,...,x)  is: 

dQ 

(i)  relative  minimum  if  -  r— r-  <  d.  <  d 
v  X-l  1  o 

d 

(ii)  relative  maximum  if  d  <  d,  <  -  r— r 

o  1  k-1 

d  d 

(iii)  undecided  if  either:  (a)-  <  d^  =  dQ  or-y^y^dj 

-d 

or:(b)  d  =  d,  <  or  d  <  d. 
v  J  o  1  — 1-1  o—l 

d 

(iv)  saddle  point  if  dj  <  min  (dQ,  -  yy) 

d 

or  if  dj  >  max  (dQ>  -  yy) . 

Graphically,  H 

.  d  *d. 
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The  method  of  Theorem.  (A. 2. 3)  can  also  be  used  to  prove  Theoren  (5.1.17) 
(since  the  required  deteminants  can  be  evaluated  as  in  (5.1.15)),  but 
is  cumbers ome . 

We  now  wish  to  investigate  the  nature  (asymptotic  as  n-*»  as  well  as 
small  sample)  of  the  critical  point  (x,...,x).  Let  x2a(b)  denote  a 

a 

non-central  chi-square  r.v.  with  "a"  decrees  o*  freedom  and 
noncentrality  "h". 


THEOPE’*: 


(5.1.18) 


I.  P  ((X,...,X)  is  a  relative  rinirmm,  or  undecided]  =  0. 

II.  P  [(X,...,X)  is  a  saddle  point] =  P  [X2.  .  (i  ^Var(M))  >  k-1]; 

U  U  K™  JL  <  o 

_  _  0^ 

otherwise  (X,...,X)  is  a  relative  maximum.  This 
probability  does  not  depend  on  n  if  y  pj*.  •  .“V  j-j-j  • 

III.  As  n->®,  P  [(X . X)  is  a  saddle  point]-*!  unless 

(in  which  case  it  is  constant  as  given  in 

II). 


Proof :  I.  Case  (i)  or  case  (iii)(a)  of  Theorem  (5.1.17)  holds  iff 
d 

(see  (5.1.13))  -  d.  <  d  ,  i.e.  iff 

K_1  1  *0 

-  i^T(Var("‘)  -  o2/n)  <  cov(n,5)  <  Var(P)  -  a2/n. 


i.e.  iff  (since  Var(R)  >  0  w.p.  1) 

C5’1-19)  "  FIT  +  (k-l)Var(R)  -  p(R,S) 


<  1  - 


o2/n 
Var(R) • 


Since  (w.p.  1)  p(R,3)  =  p  w.p.  1  equation  (5.1.1°)  fails  to  hold. 

W.p.  1  case  (iii) (b)  fails  to  hold  since  (for  it  to  hold)  at  least  one 
of  the  inequalities  in  (5.1.1?)  must  be  an  equality;  this  occurs  w.p.  0. 
II.  As  in  I,  it  can  be  seen  that  case  (ii)  holds  iff 
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Note  that  even  when  (X,...,X)  is  a  relative  maximum  it  is  not 
necessarily  an  absolute  one  (which  it  would  be  if,  e.p.,  the  system  had 
no  other  solution).  Below  we  will  find  reason  to  believe  that  the  max¬ 
imum  i  s  near  (y  ^  j  i  •  •  •  |  ^  j )  —  ( ^ 

Per  the  case  k  =  2,  Theorem  (5.1.17)  shows  (after  some  reduction) 
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that  (x,x)  is 


a  relative  maximum 


iff  (Xj-XjK  <  2o2/n 


(5.1.24)  ^undecided  (negative  semi -definite)  iff  (x^-Xjj  »  2o2/n 


(a  saddle  point 


iff  (Xj-x2r  >  2o2/n. 


Obtaining  this  result  fron  Theorem  (A. 1.1)  is  interesting.  The  limiting 
results  of  Theorem  (5.1.18)  can,  for  the  case  Y  -  2,  be  obtained  using 
(5.1.24). 

We  will  now  seek  the  !‘LF  (for  k  2) :  We  nay  (without  loss)  choose 
our  estimator  to  be  of  the  form 


(5.1.25) 


u-fi,  =  Xj  ♦  ajfXj . xk) 


t  M  =  ^iC  *  a].  j  *  •  •  •  • 

As  noted  following  (5.1.4),  we  may  restrict  ourselves  without  loss  to 
X1  —  ^  ji]  >  •  •  •  >tf[V]  )  -  xk»  ^ron  it  follows  that  we  have 


(5.1.26) 


0  <_ 


\  <  a.<  (xk-Xi)  (i=l,...,k) 

\  I  0  . 

Let  (for  1  <  i  <  k;  i  *  l,...,k) 


A£(i) 


(5.1.27) 


< 


l  ♦ 

BeSk 

B(i)»* 


'V(i)‘xrai 


x8(k)~Va^ 


u 

BeS. 


o/Zn 

J  l  a//n 

•xraii 

1  jWVakl 

i//n 
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Then  (note  that,  for  any  1  <  i  <  k,  A«  A^(i)  ♦  ♦  A^i))  from  system 

(5.1.9)  we  find  that  must  satisfy  the  system 

(5.1.28)  (xi  +  a.)A  =  XjA^i)  ♦  ...  ♦  xkAk(i)  Ci  ■  l,...,k). 

If  we  add  the  terms  of  (5.1.28)  over  i  *  1  ,  ...,k,  we  obtain  (since 
A  -  A^(l)  +  ...  +  A£(k)  for  l  =  l,...,k) 


A (Xj+ . . .  +x^)  +  (ax+. .  •+ak)A  -  A(Xj+. . .♦x^) , 
or  (since  A  >  0)  aj+...*a^  =  0.  Thus,  we  have  the 

THEOREM :  For  k  >_  2,  the  f'LE  is  Riven  by  y  ^ 

+  ai(X [!],..., Xjkj),  ...  ♦  aktXj1j,...,Xjkj), 

where  a^,...,^  are  some  solution  of  system  (5.1.28)  and  must 


(5.1.29) 


satisfy 


and 


-(xi-xj)  <  a.  <  (xJ(-xi)  (i  *  l,...,k) 


aj+. . .+ak  a  0. 


THEOREM:  For  i,j  =  l,...,k,  if  /  0  then 

(5.1.30)  d1-A1(l)  ♦  ♦  ^iV-0 

ai  “  +  ...  +  d^AJj) 

where  d.  .  *  x.  -  x.  *  -d..(i,i  =  l,...,k). 
ij  i  j  ji  J 

Proof:  System  (5.1.28)  is  equivalent  to  the  system 


a. 

l 


E  * 
8eSk 


xg(D"xrai 

o//n  , 


Voo^kl 


o/Sn 
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sn*«Ki,-v 


or  (substituting  the  ' s) 


xg(D'xrai 


a// n 


...♦ 


^B(k2lVfk 


j/»^n 


ai(A1(i)+...+Ak(i))  =  dj.A^i)  ♦  ...  +  d^.A^i) 
Thus,  the  theorem  follows.  (Mote  that  the  denominator 


(i  « 


(i  =  l,...,k) 


dyA^j)  +  ...-*■  d^A^j)  is  zero  iff  a..  *  0.) 


(5.1.31)  LEMMA:  For  the  case  k  =  2,  a^  =  -a^.  /'Iso,  0  £  aj  £  x,  -  Xj 
Proof :  From  Theorem  (5.1.30), 

djjAjU)  ♦  d21A2(l)  d21A,(l)  A2(l) 

al  =  a2d12Aj(2)  +  d22A2(2)  =  a2d1?A1(2)  =  ’*2*^  “  V 

The  theorem  follows  from  Theorem  (5.1.2°), 


LEMMA:  Let  d  =  -  Xj  >_  0.  Then  the  PLE  for  Y  »  2  is  given 

byP[l]  =  x[i]  +  al(^[l]»^[2])*  a[2]  "  X[2]  '  al(X[l)-r2]) 
where  a.  is  some  root  of 

(5.1.32) 


‘H 


d2-2a1dl 


+  e 


r2/n 


p.nd  0  a^  d. 

Proof:  By  Lemma  (5.1.31)  we  must  have  0  <_  a1  =  -a2  <  d.  Then  by 
Theorem  (5.1.29),  the  MLE  must  be  of  the  form  given  where  aj  is  some 


root  of  the  system  (5.1.28): 
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Thus, 


a*  (d-ap2  d2-2a^d 
A2^  _  .  o2/n  c2/n  _  _  o2/n 

xjn'*  • e 


and  the  lemma  follows. 


LEMMA:  For  fixed  d  and  0  <_  a^  d,  the  roots  of 


(5.1.34) 


d2-2ai d^ 


1  +  e 


o2/n 


(5.1.33) 


d  Eo 


are  (1)  *  d/2,  and  (2)  aj  =  j  *  ^jo2/n  if  d  >  /2o// n. 


Here  e  is  either  of  the  two  solutions  of 
o 

(5.1.35)  d2n/o2  =  e  coth(e/2). 

Proof :  First,  aj  »  d/2  is  seen  to  satisfy  (5.1.34).  Now,  suppose  there 
is  another  solution  of  (5.1.34),  say  (without  loss  of  penerality) 

aj  =  d/2  ♦  o2/n 

2  2  2  ? 

with  -d  n/cr  <_  c  <_d  n/ (since  0  <  <  d) ,  e  /  0.  Substituting  in 

(5.1.34),  we  find  e  must  satisfy 


or 


,2  -e 
d  e 


+ 


or  (since  e  /  0  ■>  1  -  e"e  /  0) 


> 
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d 


2 


1  +  e 


-e 


1  -  e 


-E 


S'2  .  e-^2 
ec/2  .  e-e/2 


coth(e/2) . 


(See,  e.g.,  Hodgman  (1959),  pp.  281,  427,  431,  432.)  Since  coth(-z) 

*  -coth(z),  e  coth(e/2)  is  r.n  even  function.  Now, 

.C  £  I 

lim  e  coth(e/2)  =  lim  (1  ♦  e  )  •  lim  - ■  2  lim  - «  2. 

e+o  e-*o  e-K>  j  _  e"e  e-*o  e"E 

(See,  e.g.,  Anostol  (1957),  p.  102.)  Since 

|^[e  coth(e/2)]  =  coth(e/2)  -  (e/2)  csch2(e/?) 


cosh (e/2) 
sinh (e/ 2) 


1 

sinh(e/2) 


-  (e/2)- 


1 


sinh  (e/2) 

e/2 


{cosh(£/7)  - 


the  facts  sinh (e/2)  >  0  if  e  >  0  and 

cosh(c/2)  -  5TnW!T  *  sIKRI72T!sinh(£/2)  cosh(E/2) 

1  fsinh(e) 

‘  sTnhTe/2)  [  2  e/2J 

1  T  e3  e5 
=  2  sinh (e/2) [e  +  3!  +  51  +  “* 

1  Te3  e5  e7 

=  2  sinh (e/2) [3!  +  5!  *  7!  *  '* 


-  e/2] 


g 

imply  that  -^[e  coth(e/?)]  >  0. 
plot  Figure  (5.1.36). 


Combining  the  above  information,  we  may 


Figure  (5.1.36). 


Since  coth(x)  >  1  for  x  >  0,  the  ran?e  of  e  coth(e/2)  will  be 

[2,d2-^  +]  when  e  is  in  [-d2n/o2,  d2n/o2].  Thus,  there  will  be  two 
cr- 

additional  solutions  if  d2n/o2  >  2  and  none  if  d2n/o2  <_  2. 

Mote  t^at  aj  =  0  corresponds  to  the  estimator  (\j,x0) ;  a,  =  d/2 

corresponds  to  (x,x);  and  a^  =  d  corresponds  to  (x^x^).  Consistency 
Criterion  (5.1.4)  rules  out  values  >  d/2;  thus,  in  seeking  the  r'LE  v/e 

only  consider  which  is  the  negative  solution  of  (5.1.35)  in  Theorem 
(5.1.33)  (or,  what  is  the  same,  -e  where  eo  is  the  positive  solution). 

THEOREM:  If  0  d  <_  /2  o/Sn,  (x,x)  is  the  only  critical  point 
and  is  the  '*LE. 

If  d  >  /2  o//r  there  are  two  critical  points.  <^ne 
(5.1.37)  yields  (x,x)  and  is  a  saddle  point.  The  other  yields  the  MLE 
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(5.1.38)  ('x  -  2°  o?/-,  "  ♦  o2/r), 

wlicro  cq  is  the  positive  solution  of 

(5.1.39)  1  /n2  =  c  cot'  (c/2). 


Theorem  (5.1,37)  fol lot's  from  ’•rovious  results  notably  Lemma 
(5.1.32)  for  the  form  of  the  MLH,  Lc'mn  (5.1.  S J J  for  the  solutions  of  a 
certain  equation,  and  (5.1.24)  for  the  nature  of  (x,x).  In  obtaining 
the  form  of  (5.1.38),  relations  sue*'  as 


X1  *  al 


—  O  2  , 

*  x  -  oz/n 

are  used.  Hote  that,  for  d2n/c2  laroe,"  cQ-',2n/a2,  so  that  (5.1.3?)  is 
'close*  to  (x^x^).  The  following  ler-ma  studies  toe  approach  of  cq  to 


LEMMA :  If  is  the  positive  solution  of  (5.1.3P),  then  (wit? 


(5.1.4°) 


o(n)  >  0) 


d2n 

c  =  -  -  o(r.) . 

o  ? 
cr 


Proof'  If  we  write  a  =  d2/o2 ,  then  wc  arc  interested  in  the  positive 


solution  of  c  cot’*  (e/2)  =  an.  Let  us  set  this  solution  as 


c  =  a-n  -  c  and  investigate  the  order  of  c  •  Substitution  in  the 
on  '-n 

equation , 

fa-n  -  cn 

(a*n  -  cn)  cot1,  i - - -  =  a  *n 
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It  is  of  interest  to  compare  (for  the  caso  k  ■  2)  the  likelihoods  of 
the  three  estimators  (X,X),  f  i  ]  f  2  ]  ^  *  nn^  M,'^‘  ^  "  x2_xl ' 

we  find  (see  (8. 1 . J ) ) 


o£- 

n  X(l] ,Y[2] (xl,x2) 


(5.1.42) 


.  vuiiJ. ,  «.  ffd  'jlm  *  xnui 


rm> - *-  *  — r 

o//n  J  o//n 


I  . 

I  4ovn 
I  e 


if  (M  ,u  pj)  B  (x,x) ,  the 
MLE  for  0  <  d1 2  <  ?o2/n 


1  1  o2/n  . 

?  *  2  e  ^Ufl] ,P  I”*] 


(Xj .x^) 


1  •1o2/n''' 


1  Go  o2 


1  4o2/n 


1  Co  o2 


*  2  e 


E  j  E  o 

.  ,  V  —  0  oL  —  0  0* 

if  (Mflj.M  f2])  =  lx  -  2d  n  ,x  +  2d  n  * 


\  the  MLF  for  d2  >  2o2/n. 

If  0  <  d/n/o  <  /2,  (X,X)  is  the  ML;:,  and  the  curve  of  (X,\)  has  ordinate 
1/2  when  d/n/o  =  ?fln?  -  1.67.  The  curves  of  (X,X)  and  (vf  .,v  .) 
cross  at  d/n/o  *  1.54.  At  d/n/o  =  2,  for  (y j- 1 ]  »y f 2) ^  we  finH 

2  +  \  ~  ^  ~  J  *  *01831)  *  .  50°2,  while  ror  t’e  MLE,  n  solution  of 


4  ■  e  coth(e/2)  is  approximately  cq  ■  3.8  (thus  e^/4  1  .°5)  and 
1  -  cq/4  ■  .05.  (See  Abramowitz  ard  Steeun  (ln64),  p.  216.)  Thus, 
for  the  MLH  we  find 


1 

2 


_  x2. 

'£ 

!-°  .  i 

12 

V2 

u _ 

'C 

♦  1 

4 

p 

Id  J 

tle 

4 

1  /  -.0025 

2  F 


-3.8025^ 
♦  c 


- 


,003 


♦  c 


-3.81671 


(1.010)  -  .5005. 


Note  that  Theorem  (2.1.33)  indicates  the  reasonableness  of  an 
estimator  which  compensates,  as  does  the  -'IT  =  (x^  ♦  a,  x2  -  b) ,  for 
under  and  over  estimation  with  ref  ard  to  expectation;  the  livelihood 
approach  hears  this  out. 

The  above  results  indicate  a  weakness  of  takine  a  function  of 
MLE's  to  estimate  that  function  of  the  parameters  for  a  problem  (as 
discussed  at  (5.1.3)):  namely,  other  methods  yield  different  estimators 
with  higher  likelihoods.  (In  fact,  with  t^e  other  method  t'-e  likelihood 
could  never  exceed  n/o2  ■  with  our  method  it  can  never  be  less  than 


i.MAPTEP  5.  POINT  EFTP'ATION:  MAXIMUM  LIKELIHOOD  (ML) 
AMI)  RELATED  ESTIMATORS 

5.2.  MLP.'s  FOP  NON-1-1  H  DICTIONS:  ITERATED  MLE's  (IMLE's) 


At  (5.1.3),  ve  discussed  the  problem  of  nrovidine  maximum  likeli¬ 
hood  estimators  (MLE's)  for  u  j  » •  •  •  ^  ^ »  anrJ  noted  the  Berk-7chna- 
MLE  •  most  oc  the  reminder  of  Section  5.1  was  devoted  to  a  study  of 
another  method  of  providing  ilLF's  for  ^  {•] }  *  •  •  •  f),  j  •  ’,,e  now  formulate 

this  latter  method  as  a  pcneral  inrerencc  principle  and  study  it  in 
sore  specific  cases. 

Suppose  that  0  (a  nara.noter  of  interest)  is  in  some  space  o  and 


that  wc  have  a  livelihood  function  L(0)  ( fror  o  to  R) .  flssume  that  a 
unique  MET  0  of  0  exists,  i.e.  0  c  0  such  that  T. ( e)  >  E(o)  for  nil 


0  t  0.  Let  p(  )  he  some  transformation  of  0,  and  suppose  that  r(o)  “  A. 
Then  if  q(-)  is  1-1,  0(0)  is  clearly  the  Ul.E  of  0.  If  p(  )  is  not  1-1, 
Zchna  (1066)  and  Bcr!  (1067)  hot'  pronose  to  e^'lny  the  estimator  p(0), 


whic1'  we  will  call  ti  e  Berh-Zehna-MLE . 


Z.ehna  proposes  to  use  ’(0)  sircc,  if  with  "(0)  one  associates  the 
largest  of  the  livelihoods  of  those  o'  such  that  p(e')  *  p(0),  tvis 
"induced  likelihood  function"  is  mxirized  at  p(0).  However,  as 
Dr.  Josenh  Putter  has  pointed  out  in  a  personal  communication,  "(e)  my 
also  he  a  rinimur  livelihood  estimator,  r,°.,  if  (for  some  observa¬ 
tions)  we  have  the  possibilities  as  piver.  in  Table  (5.2.1), 
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a  2  A 

then  e--2  is  the  MIT  of  o,  hut  if  r(o)  ■  o  ,  t!  en  j»(0)  ■  4  corresponds 
to  both  a  minimum  likelihood  estimator  of  0  and  a  raxirum  likelihood 
estimator  of  6. 

Berk  proposes  to  use  p(G)  since,  if  one  simply  adjoins  to  p(0) 
another  function  h(0)  so  t'-at  the  rappinr  ©-*•( r» (0) , ’* ( 6) )  is  1-1,  then 
(P(*b>h(9))  is  the  MLF  of  ("(Q),b(0)).  Rcrk  states  Ms  belief  that  it 
is  important  that  one's  estimate  maximize  the  likelihood  function  asso¬ 
ciated  with  some  r.v.;  and  since  it  is  not  clear  that  Zehna's  method 
docs  this,  Zehna  "misses  the  point."  (Mote  that  the  Iterated  f'l.E  pro¬ 
posed  below  satisfies  this  criterion.)  Perk's  reasoninr  secs  faulty 
in  that,  if  one  desires  to  estimate  p(0),  there  seers  to  be  no  reason 
to  be  concerned  with  any  1-1-izina  function  h(0).  Pather,  h (0)  is 
added  to  preserve  the  status  of  p(0)  as  ar.  "MLF."  (E.e.,  in  Putter's 

example  of  Tal  le  (5.2.1),  h ( 0)  ■  sgn(0)  will  wor!  hut  is  irrelevant  to 

2 

the  probler  of  estimating  p(0)  *  0  .) 

Let  0,  0,  0,  A,  L(0),  and  p(0)  be  as  defined  above.  (In  partic- 

ular,  we  suppose  that  0  exists  and  is  unique.)  ''e  tuen  orooose  the 

nrMNITIOM:  Consider  the  likelihood  function  oe  the  statistic 

j»(0),  sav  L  .  If  there  is  a  g  c  A  sod  that  I  (e)  >_  L  (r  ' ) 
(5.2.2)  E  P  P 

for  all  p'  e  A,  then  °  is  called  an  Iterated  MIT_  (IIUT)  of 
P(9). 


Thus,  the  IMLE  of  f>(0)  is  the  MLF  of  p ( 0)  based  on  « ( 0)  (if  it  exists 
and  is  unique) . 

Example  1.  For  the  problem  of  estimating  g(q j , . . .  ,u 
a  (w  j » •  •  •  ^  j )  *  the  Berk-Zehna  MLE  is  Xpj,...,X^,  and  in  Section 
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5.1  we  studied  the  IMLF  (i.e.,  tbc  MLF  of  based  on 

p(Xj , . . .  ,7^)  ■  (X^j ,. ..  ,Xp.j)) .  For  the  case  k  ■  2,  Klument^al  and 
Cohen  (1968)  have  compared  the  fieri  -Zehna  MLF  of  pp^  with  our  IMLF  of 
Upj,  with  reeard  to  mean  squared  error  and  bias.  Let  w  ■  Cu f 2]  ’  ufll^ 
/ 2.  They  find  that,  for  both  mean  snuared  error  and  for  bias,  the  IMLF 
is  better  for  u>  small,  and  Xp^  is  better  for  w  moderate. 

•)  2 

example  2.  Let  Yp...,Yn  be  i.i.d.  M(p,o“)  r.v.'s  '-’ith  p  and  0 

2  0 

both  unl.novn  (-<”  <  u  <  ♦°°,  0  >0).  The  M' F  of  (p,o  )  is  well-known: 

n  2 
(Y,  l  (Y.-Y)/n).  T'’en  for  estimation  of  e(p,o  )  a  p ,  the  Perk-Zehna 

i*l  1 

MLF  (which  is  Y)  and  the  IMLF  (which  is  t'  e  MLF  of  p  based  on  Y)  coin¬ 
cide.  Such  coincidence  occurs  in  "any  other  cases,  for  example  when 
our  r.v.'s  are  uniforn  on  (n,G). 


Lxarplc  3.  Let  Y^,...,Yn  be  i.i.d.  ''(p,o~)  r.v.'s  with  p 


unknown 


2  2  - 
(-»  <  p  <  ♦<*.)  and  c  known  (0  >0).  The  M'.V  of  p  is  well-Vnom:  Y. 

n  -2 

Then  for  estimation  of  p(p)  »  p“,  tse  Fer’-^ehna  MLF  is  Y  .  ,re  will 

2  -? 

now  study  the  IMLF  (which  is  t'’C  MLF  of  p  based  on  Y  ) . 

Since  (/n/o)Y  is  *'(( /T/o)p  ,  1) ,  ((*fi/o)Y)2  is  (see,  c.p.,  Fisz 

(1963),  n.  343)  a  non-central  chi-square  r.v.  with  1  c.f.  and  non- 
n  2 

centrality  X  =  — p  say  y  (X),  and  has  density  (for  x  >  0) 

2a2  1 


fj(x) 


J  -x~y- 

x  2e  2  v  (2\x)n 

■/2i  m=o“^rr 


.I  _x  J 
?  *  2 
x  e 


(/2X7) 


/2tT  m=0,2,4,... 


2  2 

- — —  e"  cosh(/2Xx)  . 

/27 


Thus,  Y?  « 


: 

n  [  0 


has  density  (for  y  >_  0) 
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fY?  (y) 


. ._Lf  . JL 

o2/n  Ma2/n 


4  ->  -2-u2 

y  • 

■  -  —  — - e  cosh 

/2n  o//t\ 


IT— J. 


-2  2  2 
Hence  (when  Y  ■  y  >  0)  the  IMLF  of  w  is  the  u  which  naximizes 


2  c2  , 
e  cosh 


i«h 


/N 


or  u2  ■  a2(Y2)Y"2  where  a2  is  the  a2  which  maximizes 


n  2 

• - V.1  * 


(5.2.3) 


?.(a2) 


2o2 


ya‘ 


cosh 


Differentiating  g(a2)  with  respect  to  a2,  we  find 

_  n  ya2  .  — ya2 

3g(a2)  n  2o2  _ fn  ..Ol  .  .  2o2  4i 


3(a2) 


2o2 


ye 


cosh 


♦  e 


sinh 


n  1 

— y— — 

o2 


which  is  <0  iff  a  >  tanh  -- ya  .  Since  tanh(z)  <  1  for  all  z  (-•  <  z  <  «) , 

[a2 

the  derivative  is  negative  for  all  a  >_  1 ,  so  we  may  seek  the  maximum  of 

n  _i  v 

(5.2.3)  for  0  <  a  <  1.  Tien,  a  >  tanh  ( — ya)  iff  tanh  (a)  >  — ya, 

a2  o2 

which  is  so  (see,  e.p.,  Hodqman  (1°5^),  n.  431)  iff 


3  5 

a  a 

a  +  • —  ♦  —  ♦ 


n 

>  — ya, 
o2 


i.e.  iff 

Since  (5.2.4)  holds  for  all  a  (0  <  a  <  1)  if  ^—y  -  1  <  0,  i.e.  if  y  < 

a  " 

the  IMLF  of  u2  is  0  if  Y2  <  o2/n.  If  y  >  o2/n,  it  is  clear  that  there 
will  be  one  critical  point  (corresponding  to  equality  in  (5.2.4))  and 


(5.2.4) 


1  +  * 

3 +  r 


a_ 

7 
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2 

that  it  will  be  a  raximun.  Thus,  the  IHLE  of  p  is 


(5.2.5) 


0  if  7^  a^/n 

2  _2  -2  2 
a  iY  jY*  if  r  >  oVn, 


where  a  is  the  root  of 


(5.2.6) 


1. 


CHAPTER  5.  POINT  ESTIMATION:  MAXIMUM  LIKELIHOOD  (ML) 
AND  RELATED  ESTIMATORS 
5.3.  GENERALIZED  MLE's  (GMLE's) 


Generalized  maximum  likelihood  estinators,  introduced  by  '"eiss  and 
Nolfowitz  (1966),  provide  (where  available)  asymptotical ly  efficient 
estimators,  whereas  this  is  not  always  true  for  MLE's  even  if  the  latter 
can  be  found.  As  noted  above,  for  the  case  of  estimating  u  ^  jj » .  • .  , 


what  is  meant  by  "the  MLE"  is  not  clear,  possibility,  the  IMLE,  is 

difficult  to  compute  and  mny  or  mny  not  possess  desirable  properties. 
Most  classical  MLE  theory  assumes  i.i.d.  observations  and  is  therefore 
not  applicable  in  our  case,  since  the  IMLE  is  in  this  case  the  MLE 
based  on  non-i.i.d.  observations:  the  ranked  data.  Tie  theory  of  Weiss 
and  Wolfowitz  (1966)  allows  for  more  pencral  situations,  although  most 
of  their  applications  are  to  i.i.d.  "non-rcpulRr"  cases.  (Corrections 
to  V'eiss  and  Wolfowitz  (1966)  are  contained  in  '"eiss  and  ''olfowitz 
(1967a),  in  Weiss  and  '"olfowitz  (l°67b),  and  below.  An  additional 
example  is  piven  in  ''eiss  and  '‘'olfowitz  (l°67c).) 

We  first  summarize  t'c  results  of  Weiss  and  '"olfowitz  (1%6)  for 
the  case  k  ■  2. 


PrcINITION:  Let  0  be  a  closed  region  in  R2,  0  c  0  with  0  a 
(5.3.1)  closed  reoion  such  that  every  finite  boundary  point  of  0  is  an 
inner  point  of  0. 


(5.3.2) 


DEFINITION :  For  each  n  let  X(n)  denote  the  (finite)  vector 
of  r.v.'s  of  which  the  estimator  is  to  be  a  function. 
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(5.3.3) 


(5.3.4) 


(5.3.5) 


DEFINITION:  Let  !<n(x|0)  be  the  density,  with  resnect  to  a 
o-finite  measure  un,  of  X(n)  at  the  point  x  (of  the 
appropriate  space)  when  0  is  the  "true"  value  of  the  unknown 
parameter. 

DEFINITION:  Let  r  =  (r^.r^)  be  fixed  and  positive. 

{Z  i (X(n)  ,r)  >Zn2(X(n)  »r)}  is  a  sequence  of  GMLE's  if,  for  eac’’ 

0  =  (0j,02)  c  0,  (/*,')  and  (B1)  below  are  satisfied. 


CONDITION  (A1):  There  is  a  sequence  of  positive  constants 
(Ljfn)  ,k?(n)}  such  that  k.(n)-n»,  ^f*1)-*®,  and  a  function 

L(yi,y2|0)  such  that  L ( •  j 6 )  is  a  continuous  d.<\,  and,  for 

any  y  =  (yj,y2)  and  an^  integers  aid  h2 


lim  P 
n-*» 


Vl  h2r2 

61+  k1(n)'»fl2+  k2(n) 


kj  (n) 


hr 

7nl'er  FJTnT 


1.  y 


r 


k2(n) 


V? 

Zn2'62'  h2(nj 


--  ^2 


*  L(y1,y2l01»e?) 


CONDITION  (B'J:  For  any  intepers  hj,h2  there  exists  a  set 
Sn(0,hj,h2)  in  the  space  of  ^(n)  such  that 

(5.3  7)  lim  Pq  [X(n)  e  S^e.hj.h.,)]  -  1  (i,j-0,l), 

ij 


where 
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(5.3.8) 


Chj+i) rx  (h2+j)r? 

“ij  =  [91  +  kj(n>  »  6?  +  k2(n) 


and.  there  exist  sequences 

(5.3.9)  (an.  (X(n),0,hlfh2)) 

of  (two-dimensional)  r.v.'s  such  that,  as  n-*». 


(ij  -  0,1) 


a  . .  =  (a  .  ..  ,a  .  ._)  converges  stochastically  to  zero  when 
mj  v  nijl*  nij2'  1 

is  the  parameter  of  the  density  of  X(n),  and  such  that, 


(5.3.6) 


whenever  X(n)  e  Sn(0,h^,h2),  we  have  the  followinp:  Let 


(5.3.10)  M  =  nax{Kn(X(n)|ot.j),  (i,j  =  0,1)), 


(5.3.11)  m  =  (n  n2)  = 


(h2+l/2)r2 


'1 


(h1+l/2)r1 
+  kjfn)  ’  62  +  ^fn) 


Then,  where  "(a<b,  c  <  d)"  means  "(a  <h,  c  <  d)  or 
(a  <  b,  c  1  d) ," 


(5.3.12a)  M  =  Kn(X(n)|a0{))^ 


nOOl  „  _  n002 

7  ,  <  m,  +  ? — f — c-,  7  _  <  +  ; — j— r- 
nl  1  hj  (n)  n?  2  k2(n) 


(5.3.12b)  M  *  Kn(X(n)|a01)~> 


,  an011  _  an012 

7nl  <  F1  +  kj  (n)  ’  r2  EjTnJ 


(5.3.12c)  M  =  Kn(X(n)|a10)-> 


„  _  _  Snl01  „  _  .  anl02 

7nl  1  ^TnT’  >2  2  k?(n) 


(5.3.1 2d)  M  =  Kn(X(n)jon)*> 


,  _  anlll  „  .  anll2 

1  .  >  m.  +  \ — 3— v,  7  _  >  +  r— y»~T 
nl  1  kj(n)’  n2  2  k2(n) 
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TIIEOPF.M:  (Weiss  and  "'olfowitz)  Lct{Zrl(X(n),r),Zr2(X(n)  ,r)} 

be  a  sequence  or  Gf'U:,s.  Let  {T^}  be  any  seauence  of 
estimators  of  6  such  that,  for  fixed  r  =  (rj,r2)  an(^ 
interers  hj,h2 


lim  P 


r  ri 


ki(n)(Tnre1)  if 


-  T<k 2(n><Tn2-e2) 


Note  that  on  n.  78  of  Weiss  and  Wolfowitz  (1966),  condition  (B1)  is 
mis-stated1  therein,  in  (3.13)  through  (3.16)  (corresnordin"  to  our 
(5.3.12a)  through  (5.3.1.?d)  above) 

Koor  an011  ’  anl01 *  anlll:  an002,  an012’  anl02’  anll2^ 
should  be 

^an001  an011  anl°l  anlll  an002  an012  anl02  anll2; 

-k^n)’  k^n)’  kj  (n)  ’  k^n)’  k2(n)‘  k2(nf  fc2(n)'  k2(n^  * 


Examination  of  the  modification  of  t’>e  proof  of  pp.  73-74  of  '''oiss  and 
Wolfowitz  (1966)  used  for  the  proof  of  their  Theorer  3.2  (Theorem 
(5.3.13)  above)  shows  that  without  this  chanpe  the  quantities  a^^ 

multiplied  by  the  normalizing  factors  kj(n)  and  kj(n)  would  occur,  and 
would  not  necessarily  converge  stochastically  to  zero  (under  the  appro¬ 
priate  parameters).  In  their  multi-parameter  examples  VI,  VII,  and  VIII 
Weiss  and  h’olfowitz  (1066)  seem  to  satisfy  the  corrected  (B ' ) .  (In 
example  VIII  this  is  not  as  clear  as  in  examples  VI  and  VII.) 

We  now  investigate  the  application  of  these  results  to  the  estima¬ 
tion  of  u , . . .  ,u ^ j .  For  k  >  2  we  now  choose 


(5.3.14) 


rX(n)  »  (X(1],...XV]) 

K  (x|e)  =  K  (x  |u )  «=  f^  y  (x.,...,x.) 

n  n  a  |*i]  »  •  •  .  >  ^  1  * 


5  f-v 


7  »  •  •  •  »^i,) 

[k] 


Y  Y  v."i  >  •  •  • 

xrij,...,x„.,  i  K 


\u  =  Lebespue  measure  on  R\ 

\n 

We  would  also  like  to  choose  0  ■  (u  :  u  c  ,p  j=u  » •  •  •  )»  0 


(which  would  satisfy  (5.3.1)),  but  by  Theorem  (B.2.10)  this  would  not 

c 

allow  satisfaction  o^  condition  (A*)  (essentially  because  ij  c  Gr,[Q(jf)] 
would  not  uniquely  specify  the  limiting  distribution).  ^bus,  we  fix 
n*  >  0  and  choose 

f 

iO(n*)  =(u:  u  e  G,  Uj.-Pk_j  >  n*.  Vj,_1-Ujc_2  1  n*,  ...  i  n*) 

(5.3. 15)/ 

10  =  G(n*/2)  . 


(Although  our  results  below  would  hold  if  we  simply  excluded  the  bound¬ 
aries  of  our  desired  G,  that  set  would  not  be  closed.)  Since  our  results 


RO 


lack  real  dependence  on  n*,  we  have  essentially  only  eliminated  the 
boundary  (where  equalities  exist). 

For  k  ^  2,  consider  the  sequence 

(5.3.16)  {Znl(X(n),r) . ZnI.  (X(n)  ,r)  )  -  (Xfl) . X^) 

with  r  *  (Tj,...,r^)  fixed  and  positive. 

THEOPF.M :  For  k>  2,  condition  (A')  (or,  more  nropeily,  its 

(5.3.17)  pcncral i zation  to  k  >  2)  hole's  for  the  sequence  (5.3.16)  for 
arbitrary  r>  0,  with  kj(n)  «  k^(n)  »  /n/o. 

Proof:  This  follows  from  Theorem  (P.2.8). 


LCf'l-lA :  Let  h^  and  h?  be  any  integers.  Choose  Sn (p  ,hj .h^) 

■  *kn  [,)-«„  i*(,|  i  "[,]«„•  u(2)-cn  iff21  -  u  (2]+cn>’ 

(5.3.18) 

where  ■  o/n  (0  <  6  <  1/2  fixed).  Then  (for  i,j  *  0,1) 

lin  n  [X(n)  c  S  fti.h  ,h7)J  -  1. 
ij  n  1 

Proof:  Py  (5.3.8),  here  a.  ^  *  (p  [l  ]  +  P',i*i)r1o/*/n,u  p^  +  (h2+j)r2o//n) , 
and  (setting  aj  *  (hj+i)^,  a2  =  (h2+j)r2) 

V.(X(n)  c  5i,6‘-V,2)'  3  P,  *ao//n[“  m'°/n4  --hi] 

1  u  j.  ^o/n5  (i*l  ,2 


(5.3.19) 


♦ao//n 


1 

2 


<  Xnru[irai0/^ 

o/  /n 


a//n 


1 

2 

n 


PI 


However,  by  Theorem  (B.2.t)  the  random  quantities  of  (5.3.10)  approach 
a  joint  limiting  distribution,  while  the  respective  upper  and  lower 
limits  on  those  quantities  tend  to  ♦».  (In  fact,  the  result  is  proven 
for  any  fixed  a  ■  (a^  .ap  and  not  just  for  ( (h^ijrj ,  (h^+ j ) r^ ) . ) 


As  noted  in  the  proof  of  henna  (5.3.15),  for  our  case  we  have 
(for  i , j  «  0,1) 

(5.3.20)  a.j  =  (Mfi].(h1.i)r1o/^,  u  f?)*(lyj)r,a/rf0. 

LOW :  If  =  2,  then  (for  i,j  =  0,1) 


r2h2  r2h2 

Jr i_  4  ~2n2 

K  (x|a.  .)2tt  2i  e  2 
n  1  ij-'  n 


„  /l'U  flJ  •  n  1  2  ^  1  2 

a.eril'^r-i(h>*7,,r'*r^_-3fV?5)i 

*  B  C 


(5.3.21) 


b  'e 

where 


.Xl'Ur2]  .,L  l.„  ,  .X2'ufl]  .  ,t  1.,  , 

r2 >-7T  '  ,(  2  *  ?J,r2  *  Tl1~7r  '  ltV  2l)ri 

o//n  o//n 


1 

Vri]’ 

1 

X2"U  [2j 

"2 

*  c 

o//n  _ 

2 

o/Sn 

1 

^[2] 

i. 

1 

Vm' 

*  2 
=  e 

i  . 

2 

o//n 

r  „  r  h  *211121 

1  1  o/fn  22  a  I /H 
e  e 


L  xrlJr2i  ,  x2'urn 

r,h, - -  r.  ^ - 'z~i- 

a/J n  a//n 

e  e 


Proof :  (Note  that  a'  >  0  and  f: *  >  0  involve  only  o,  n,  x^ ,  x.,,  p  ^ 


u f2],  rj.  rT»  and  h^,  and  not  i  n^d  j.)  From  (5.3.14),  (5.3.20), 
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m 


xi"urij,, 

— — V"J  ? 

,  o//n 


ri  *vum  ** 

(lit  +  i)r, -(h,*i)2--  ♦  -r_IJ-(h2*j)r2-(h2^j)2~ 

a/A\ 


♦  e 


1 

xru_[?i' 

i 

1 

”  2 
e 

.  a//n 

'  2 

.  o//n 

i— !^(h2*j)r2-(h2*j)!,-  *  -i-Uldij.ilrj-Oi,*!)2/ 
o//n  o//n 


r2h2  r2b2 
11  r2  2 


’  ' 

2 

1 

xi‘M  r  ii 

1 

X2‘W  f?l 

*7 

e 

o/i/n  ; 

7 

c//n  , 

-  i2?1  r2(Vj)~~^  -j2*-  -JV 

J  1  0/^  ^  1  1  Z  “  o/^ 


♦  e 


2 

’  ' 

li;gL2j 

1^ 

(il 

o/Sn 

2 

, 

x.  -w  r->i  Tl 

-IV! 


X2*'J  pi  ,ri  , 

-iv  -‘v. 

e  ; 


r2h2 

11 


r2h2 
2  2 


a'e 


o/*^ 


i2ri  ,ih  r2  r  *22121  ./j.  .jh  r2 
i  2  injij  r2;  J  ?  j  2t? 

a//  n 


r  ,2 i  _jh  r2  r  iV!ni  „.2!i  .ih  r2 

r23  ,  r  3  2  3  2T2  r  ,r  2  1  11 

a/  vn 


+  b'e 


j//n 


(n  ri 
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LEMMA:  There  exist  a^j  and  a^^  nfly  depend  on 

X(n),  u,  hj,  and  h^)  which  converge  stochastically  to  zero 
when  a.j  is  the  parameter  of  the  density  of  X(n)  (i,j*0,l) 

such  that,  if  X(n)e  (u  ,h j  ,h.,)  and  M  ■  Kn  (Y  (n)  |o^ ^ ) ,  then 


(5.3.22) 


xnrj2i 

o//n 

(5.3.23)  • 

an'1 

v.i2r_ujii 

o/  A\ 

(ii)  for  i 

,Xj  SL>± 

[  rjjjioi 

o//n 

(5.3.24)  « 

and 

X f2]~g  r 2 1 

n/Sri 

(iii)  for  i , j  =  l,n 

ym"u  m 

o//p 

(5.3.25)  ■ 

and 

Xf?l'tif2) 

o//n 

°Y  2)rl  *  >1 


(h?+  2^t2  * 


noil 


2t  T 


n012 


'nlOl 


<  (’•  ♦  7)r.  ♦  a 


2  nln2 
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(iv)  for  i .  j  ■  l_j_l 


(5.3.26) 


XLU.'.  U1  >  (h  .  i)r  .  , 

o//n  1  1  21  nlll 


and 


X[21'“f2)  ,v  1, 

\r  '  CY  7)r?  *  anll2. 
o//n 


Proof :  (i)  Case  i,j  «  0,0.  For  simplicity,  write  x  for  X(n),  for 

X[l]  ’  X2  ^or  yp]  *  u  i  ^or  y  [l]  -  anf^  u  2  ^or  u  [2]  ‘  S*nce 

Kn^Xla0(P  -  Kn^xK(P’  by  Lonna  (5-3*21)* 


a'  +  b'  >  a'e 


*1*1 

^a//n 


-  (hj* 


Vgl 


. ,  o/^r 

b'e 


(5.3.27) 


r, - (*ij+  |)rj 


Xl-Ul 


>  a'e  l°/S 


,  ,  *a/Sn 
♦  h 1  e 


■  <V  T>rl 
-  O',*  jlr* 


V“l 


since  x^.  Thus 


‘i  /  r-  '  (hl  +  2)rl 
1  >  e  °  n  and  (taking  logarithms) 


0  >  fj 


*1^1 


i//n 


-  (Y  2>rl 


so  that  (since  rj  >  0) 


X1 _Vi  1  1 

(5.3.28) 

o/v n 

!‘/e  may  (for  example)  take  anQ01  =  ^  and  thereby  satisfy  t’’e  first  part 
of  (5.3.23). 

Since  Kn(x|«00)  ^Kn(x|a01),  by  Lerna  (5.3.21), 
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x.-ti 


X?^2  1  2 
r2  '  h2  *  2  r2  c 

a'  ♦  V  >  a'c  °//n  .  b'e  o//n 


-2“  •  (h2  *  7)T2 


(5.3.29) 


>  a'e 


x')"|l2  1  •> 

r2~Tf-  ■  (h2  ♦  T>i 

o//n 


or 


,  b' 

1  ♦  — >  e 
a ' 


(5.3.30) 

Now,  by  the  definitions  or  a'  an<'  b', 


o//n 


-  (\  ♦  i)r2 


(S.3.31) 


0  <  — - 

a ' 


1 

Xl‘U2 

2 

1 

pw 

2 

■  2 
e 

o/rfT  ; 

‘  2 

a/^n  J 

1 

fXl"P  lj 

2 

1 

[V^j 

T 

e 

o//n 

2 

Lo/*^n  . 

u  xru2  u  Vgi 

r  b - r  h - 

o//n  o//n 

e  _ e _ _ 

u  Xl^l  ,  X2*g 2 

r^r  r,h,— - 

o/*^  o//n 

e  e 


Vui  4  “r"2 

o/v^n  o//n  J 


>2 


X2‘lJ2  g2‘ul 


\  2 


c//n  o/*^ 
2 


V. 

2 

1 

x7-u2 

,0/v^  ; 

2 

L°/^  , 

(ri‘1rr2h2) 


/n. 


X2X1 

o//n 


(u  ?-u  L)  (x,-Xj)  (r1h1-r2^2)-  ofx2*xP 


=  c 


-  n-(x2-x1)((u2-u1)-  7i(V1-r2b2)} 

cr  /n  ... 

e  if  u2  >  p  j 


X2  X1 
o/i/n 


frlVr2h2) 


if  u2  =  u, 
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Since  w,  >  g  j ,  from  (5. 3. 30)  and  (5.3.31)  we  find  (taking  loparithrrs  and 
simplifying)  that 


(5.3.32) 


x2~u  2 


r  .<  ^)r,  .  J.- «n(l.  £ r ) 

o//n  2 


1  b  ’ 

we  now  wish  to  show  that  the  choice  a  ....  =  —  Jtn(l+  is  effective. 

nt)0?  r„  a’' 


(Here  we  use  the  fact  that  ang02  ray  d.enend  on  u,  as  well  as  on  X(n), 
hj,  and  h^.)  Since 


lXf2l^  [2]  I  +  lXl 


r  i  ]  ~w  r  i  ] 1 ' 


for  any  c  >  ®  ^  ^  f  2  ] f  2  ]  ^  2*  ^X[l)"u[l]^  <  2^ 

i ^  [ 2] -X  [  1  ]  ^  ^  f  ’’I  [  1  ]  ^  I  <  e  ^  that 


(5.3.33) 


paoo[! cxf2rxfii)'(lJ,?rlJ ri]}  1  <  el 


*  Pa00f|X[2]n'[2]1  "  C/2’|Vm"M[ll 


'Urn  I  <  e/21- 


By  Theoren  (B.2.8),  as  n-*» 


(5.3.34) 


p0oo'ix[i)-“ri]i  <  E/2>  *  %„(-«/*  < 

p%0f-hiri-  lr-‘  4<"rir  fi)'hiri0/’/")  'n-Vi’41' 


a  similar  result  holdinp  for  Ry  Lemra  (B.2.1),  the  r.h.s.  of 

(5.3.33)  -*-1  as  n-+®,  so  that  the  t.h.s.  must  also  -*-1  as  n-»®.  Taking 
c  «  c '  Oj  j2]'u  [i]^  w*t*1  0  <  e*  <  1,  this  means  that  as  (n-*50) 

pc.0J(1'e')Cj [2]'wfi])  <  xf2]_x[i] <  (1+r,)6jf2r’Jri]^'"1* 


(5.3.35) 
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Using  (5.3.35),  noting  that  ^-Xj  >  an<*  ta^inp  n  1  (r^hj-rjl^^a2 • 


-a 


•6/(p 2-u  j)2»  it  follows  that  the  exponent  an  (say)  of  b'/a'  *  e  n  in 
(5.3.31)  is  such  that  for  all  finite  x  wt 
Then  it  can  be  shov/n  (successively)  that 


(5.3.31)  is  such  that  for  all  finite  x  we  have  P  [a  <  x]+0  as  n-*». 

a00  n~ 


(5.3.36) 


Pa  PiJ- 
00 


1,  x  >  0 

0,  X  <  0 


(5.3.37) 


p 

in 

-a 

i  n 

1+e 

J 

“oo 

.. 

J 

1 

1,  x  >  0 
0,  x  1  0 


From  (5.3.37)  it  follows  that  our  a  converges  stochastically  to  zero 
under  “00- 

(ii)  Case  i,j  =  0,1.  Since  Kn(x|aQ0)  <_  Kn(x|ctpj),  by  Lemira 
(5.3.21), 


r, - (h_+ 

i  ,  ,  ,  a//n  "  .  ,  ia//n 

a'  +  b'  <  a'e  +  b'e 


x.-u,  i 

r2 - "C’12+  2)t2 


(5.3.38) 


x7-u-  . 

t2~~P‘  ’  ^2+  2)r2 


<  a'e 


Xi'Pi  -i 

T  . L .3  _fh  ♦  Ilr2 

2  ,  r-  1  2  2jr2 


♦  b'e 


:/Sn 


X2'w2  ,,  K 

-  (h2+  2)r2 


since  Xj  <_  x2.  Thus  e  L 


o//n 


x2~h  2 
[p/'/n 

(5.3.39) 


(h2+  2)r2 


>  1  and  (taking  logarithms) 
>_  0  so  that  (since  r2  >  0) 


x2'w  2  1 
—  i<V?r2' 


j/^n 
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V.'e  may  (for  example)  take  angj->  =  -  —  and  thereby  satisfy  the  second 
part  of  (5.3.24) . 

Since  Kn(x|af)j)  ^^(xjcijj),  by  Lepra  (5.3.21)  we  have 


r  _?__.2  _(h  ♦  -)r2 
2 _.r~  1  2  2  r2 


Xl~u2 


a'e 


j//n 


2  .  r  '•‘‘2  2J 

.  .  o//n 
♦  b  •  e 


U2 


-(h.+  j)t * 


xl"u  1 


<hi*  ?)ri  *  Tirr  -(V  ?>i 


>  a'e 


.  la//n 


o/Sn 


xrw2 

-o/ A' 


I  1  1 

-(h2+  ~)r22  ♦  rj——  -(h,+  ~)r2 


+  b  'e 

This  can  be  reduced  as  follows: 


j//n 


1  2'*1 


a'e 


X2^2  fh  1,  2 

?  ,j-  "  h2  2  r2 

o//n 


Xl*u2 


2o //FT  ’(h2+  ^ 


♦  b  'e 


x2iJi 


1'  2 


ri-7T  -P>i* 

l-e  o/y" 


X.-iJ 


Vu2 


l'-2 


>  a’e 


xrx2 


rl  .J-  -(hr  2")rl  r2  ‘ .r  '(h2+  2)r2 
o//n  o//n 


,  b'  o/AT 

1  ♦  —je 

a' 


Xl'ul 


V"!  _n.  1^.2l 

Xj-U 

- 1 

5* 

k- 

kc 

I  \ 

D 

rio/AT 
>  e 

f  VX2 

r  X2-U] 

I  (h,4>rl  *  r-,n1 

o//n  1 


r? 

.  b'  o/Ai 

1  ♦  — ,  e 
a 1 


l-e 


o/ »An 


Since  >  Uj,  use  o*"  (5.3.31)  reduces  this  inequality  to 


o//n 
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1  Chj+  y)rj  +  — £n-  1+e 


— (x2-x1){  (u  2’Uj)-  ^zO^hj-r^)} 

*/n 


(5.3.40) 


/n ,  J  *2^ 

r— (x  -x9)  r. — - 

2  0  1  2  ,  la/Sn 

;  1-e 


x,-u  i  i 

T\~7r  -Ch!+  2)rl 


<V  -)ri  +  --4nl+e 


1  [  -  ^X2-X1){&J2-'J1)-  ^(rlhl*r2(h2+]))} 


X2  ^  1 
*  r  i  _  _  _ 

*c/^ n 


'V  i>ri 


In  order  to  show  that  the  choice  of  a  ...  as  the  second  term  on  the 

nOll 

r.h.s.  of  (5.3.40)  is  effective,  we  will  show  t^at 


(5.3.41) 


1 ,  x  >  0 

Pa  [‘Vll  -  X] +  ‘ 

oi  o,  x  <  n. 


This  implies  that  a^^  converges  stochastically  to  zero  under  a^. 

(To  show  the  inequality  of  (5.3.24),  an011  should  actually  he  taken  as 
(e.p.)  the  above  plus  -.)  ”ow ,  if  is  replaced  by  nr'*  h^+l 
replaces  h^,  then  t’’e  sane  proof  that  yielded  (5.3.35)  yields  (with 
0  <  c'  <  1) 

(5.3.42)  [  (1-c  ' )  (y  j  i  j )  <  ^[2]-^[l]  <  ^+G  ' )  Cp  ”P  dj )  ]"*  1 

as  n-x».  Psing  (5.3.42),  noting  that  ^2-x^  >  0,  and  taking 
n  (ri,1i'r2(h2+l)) 2a2^/(P  2"P  i)2  6  >  1,  the  exponent  of 
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(5.3.43) 


-  —(x^-Xj) {  Cj  2-y  ,)-  7=(riVr2 Ch2+1))} 

rr^  '  »/n 


is  such  that  (as  n-*») 


(5.3.44) 


['V  5.  xl  + 


01 


1 ,  x  >  0 

0,  x  <  0 
“ 


In 


B 

n 


x0-y 

o77T  "(hl+  7)1-1 


I  „  1  \  _2 


<V  I)rl  ^'WW 


-e 


e 


A 

n 


(5.3.45) 


-dy  J)»f 


=  -e 


■2(x  -XjHC^-u.)-  — (r1(h1+l)-r2(h?+l))  > 
32  /n 


/n,  > 

ri-?(xi-“i) 

•e 

the  middle  exponential  term  tends  stochastically  to  zero  (under  anj  as 

n-*»)  as  did  A^,  since  it  is  wit1,  hj  replaced  by  h^  +  1,  and  the  first 

exponential  term  is  a  constant.  By  Theore'*1  (B.7.8), 

exp(rj/n(X^j-u  j-jj)/o)  has  a  non-depenerate  limiting  distribution  since 

(for  any  x  >  0) 


‘01 


rl  a CXf 1] m 5 


<  x 


=  F 


01 


rl  a(X[l]*w  rip  -  £n  x 


=  D 


‘01 


/-  1/r. 

■-  *"  x  -  Vi; 


(5.3.46) 
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It  then  follows  that,  as  n-*», 


(5.3.47) 


t 


x]- 


1. 

0, 


x  >  0 
x  <  0  . 


By  (5.3.44)  and  (5.3.47),  1  *  *  Bn  converges  stochastically  to  1 


a„.  as  n-*»,  and  since  a  =  — Jln(l+A  +B  }  it  follows  that  a 
ni  ’  nOll  fj  n  n 


nOll 


converges  stochastically  to  zero  under  a^j . 

(iii)  Cose  i,j  =  1,0.  Since  Kn(xjn^)  >_  Lemma 

(5.3.21), 


Xl^l  f\\  1>  2 

rl  ,r  ‘(hl+  2)rl  ‘1  .  r 

°//n  +  h.e  °//n 


x,-u,  , 

r, - (hj*  i)  i*j 


a'e 


>  ae 


x .  “U  i  i  —  - 

rl  — ‘(V  2)rl  *  Tl~T  ‘'V  2,r2 

o//n  o//n 


r  _ Z.  .  rv,  +  — )r2  +  r  — _ -  -W2 

r?  ,  n  1  2  2)T?  rl 


VW1 


,  ,  a//n 
+  b '  e 


i/Sn 


-  <V 


Xl'lJl  k  2  ^2  H2  k  2 

Ti  -0>,*  y)rf  r2-—  -(*•,.  ?)r| 


V2 


>  a'e 


j/v^n 


o//n 


1  +  — ,e 
a' 


VX1  VW2  k  2 

.  ,  rlT7-  r?-7"(h2+  2)r2 

b'  o//n  o//n 

>  e  : 


X2*U2  „  1,  1  „  4 

o//R~  '  2*  2)r2  F^" 


1  U 
1  +  — ie 


.  X2~XI 
^o/i/n 


under 


03 


We  will  now  show  that  the  choice  a 


nl02  r, 


-Jin 


VX1 


.  h'  o//n 
1*  — ;  e 
a 1 


is  effective.  Since  u2  >  by  (5.3.31) 


Vxi 

r  2 - — 

rc  i  ao\  b'  a//n 

(5.3.48)  — ,e 


-  —  (x?-x1)((m2-p1)-  —  (rjfhj  +  D-^h,)} 
o2  “  /r\ 


and  the  argument  of  (5.3.42)  through  (5.3.44)  can  be  modified  to  show 

that  this  converges  stochastically  to  zero  under  a^n  as  n->®.  The 

result  for  a  in-  then  follows, 
nlu/ 


Since  Kn(x|ain)  >_Kn(x|a00),  by  Le^ma  (5.3.21) 


a'e 


-(hi*  ?)ri  rirr/“  -<Y  i)ri 


vui 


1.-2 


.  b'e  o//l’ 


x,-y,  , 

T  +  -)r2 

ri  ,r  2jri 

,  a//n 


b'e 
( 

r 

a'+b'e 


>  a '  +  b 1 


xj:!i 

lo/*/n 


>  a'  ♦  b' 


^1  1  1  1  L.  |  -I 

-~rr  -  (Y  2)rl  +  Yn<1  +  5V)  -  r^n 

o//n  1  1 


VX1 


’o'  *o//n 


1+  — :e 
a' 


The  efficacy  of  a  ...  =  —  £n(l+  — )  -  £n 
7  nlOl  r,  a  7  r 


1 


1 


x?'xl 

r - - 

,  b'  a// n 

1  +  -  -.  e 
a ' 


is  shown  by  a  modification  (allowinr  for  n^)  of  the  proof  for  ?ni02 


above . 
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(iv)  Case  i ,  j  =  1,1.  Since  K^xla^)  <  ^n(xlr',ij)»  by  Lemma 
(5.3.21)  and  the  fact  that  Xj  <  x^ 


x2~^  2 


-  <V  J)T2 


X2~U  2  ,L  1,  ?  Xl'w2  /t  1, 

r - (h-  +  )r^  r - (h  +  _)r 

,  ,  2a//n  2  2  2  ,  2o//Z  2  2 

<  a'e  +  b'e 


(5.3.49) 


VM1  ,,  K  2  Vp?  r.  .  1,  2 

r  -(h  *  -)r  *  r2— —  -Oy  J)T2 

<  a’e  a//n  a//n 


X1'U2 


-(h?*  ^)r*  .  r,4-l  -Pi,*  j)rf 


+  b'e 


xrwi  k  2  “2  h 

rr  -(h  *  j)r  *  r  -  — 
<  a.e  o/^n  o//n 


X2*1J  2  ,,  K  2 

"TT7  -(h?*  ?)r2 


t~T1  ~  (h2+  ^r2  +  rf~-  “(V  i)ri 


]  2'*1 


+  b’e 


so  that  (utilizin'*  the  first  and  last  lines  above) 


(5.3.50) 


xi  i  i  i 

*(hl  +  T)rl  rl~77^  -(hl+  2)rl 

1  <  e  °An  +  l  e  a//" 

a’ 


rg£  -(h>*  ?,r’ 


r 

b’  *o//n  | 


(5.3.51)  X1'U1  ,,  1. 

~77r  -  ( V  2^*1 

o//n 


X2X1 
r - 

1  .  .  b'  o//n 

— in  1+  --e 
Tj  a' 


csi  r  i 


The  efficacy  of  a  ni  *  ■  — £n 

”1 


!♦  — |e  lo/^ 
a' 


is  shown  hy  a  modifica¬ 


tion  (allowing  for  a^)  of  the  proof  for  a 

Since  Kn(x|a10)  <  Kn(x|an),  we  obtain  (as  with  (5.3.49)) 


Vgl  n,  U  2 

Ti~7?~  "(V  2)rl 

a-e  o//r 


Vwi 


,  ^a/v'n 
<  a'  e 


-Chj-  V77i-(hl*5,rl 

.  ,  o//n 
♦  b '  e 


X?nJl 


,  ^o//n 

<  a 'e 


x.-g.  .  1 

- <V  Vi  *  Ti~r  -(lv  51rz 


~o//t\ 


r  LOIl  _fh  .  V  *  r  -221  -(■>  .  i)r* 

T2  ,r  in2  2Jr2  rl  ./-  1  1  1 

o/fn  o//n 


x,-u 


+  b  'e 


Vi 


X--U 


r, — 


•1  ,r  ‘<Y?)rl  r?77  '(,V  ?,r? 

o//n  o//n 

<  e  e 


a ' +h  '  e 


x->*xl 

1 0/ »/n 


and  (as  with  (5.3.50)) 


Vy2 
r2  ~  (V  pr2 

o//n 


r  V_*l 

1+  b_'„  l°/^ 
a  •' 


1  <  e 

The  rest  of  the  proof  is  similar  to  that  of  the  first  part  of  case  (iv) 
after  (5.3.51). 


THEOREM:  For  k  >  2,  condition  (B')  (or,  more  properly,  its 
(5.3.52)  generalization  to  k  ^  2)  holds  for  the  sentience  (5.3,16)  for 
arbitrary  r  >  0. 


r 
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Proof :  Condition  (B ' )  is  riven  at  (5.3.6).  Its  first  renuirement , 
(5.3.7),  is  satisfied  hy  (the  renerali zation  to  k  >  2  of)  Lemma  (5.3.18) 
The  remainder  of  its  rcqui regents  are  satisfied  (for  t'  e  case  k  «  2)  by 
Lemma  (5.3.22).  we  will  no"  show  that  these  re^ainino  requirements 
are  satisfied  when  k  >  2. 

As  at  (5.3.21)  and  (5.3.20),  for  ij,...,!^  *  0,1 


K  (x|u  )  =  fy(u)  cr 
n  x[l] . [k]  1 


»  (x.,...,x.) 

V  n  1  1  * 


(5.3.53) 


Thus , 


Se**,  o//n  I  a/Sn 


(5.3.54)  air-.ik  "  (Ufl]  +  (hl+il^l0//n’***'M[vr(\+ir)rV°//n)l 


K  x  a. 


n  |  lji2. 


(/ 2vo//n)^e  2 


.+  -y~ 


,/j-,k  T  2 

=  (/2 n)  e 


—  k 

?Tl)  • 


'  1  e 
ecSk 


1  V  hui^m^yw74 

i  L  /— 

j  =  l(  o//n 


r!hl  rkht 

..........  _r_ 

*  e  L 


07 


M 


xjmlD2 


o//n 


(5.3.55) 


r2h2 

11 


+ . .  .♦ 


Tlhl 

k  k 


k 

l 


2  r  J  =  1 


l  «- 

6eSk 


- 1 

x30)'11m 

♦h.r 

2 

Q 

2l 

j  J  a //a 

k 

l 

J*1 


>J  0/4T  J  J  2  J  3  2  J 


.1 

l  a'fBje*"1 

8cSk 


^  ^  o/Sn 


J  J  2  V  J 


where 


1 

Xeu)"wrj] 

th  r 

■  2 

. 

o/Sn 

^  ^  o//n 

k 

.1 

(5.3.56)  a'  (8)  =  e^1 

Vftule  for  the  case  k  a  ?  there  were  2!  =  2  terns  in  the  final  summation, 
there  are  now  k!  terns. 

As  there  were  22  =  4  parts  to  Lenna  (5.3.22),  there  arc  2  parts 
here.  We  will  ~ive  the  proof  for  the  part  corresponding  to  (5.3.23), 
since  it  is  indicative.  I.e.,  in  the  case  ij,...,i^  =  0,...,0, 


(5.3.5") 


X  r  -i  i  "W 


01 


A  r-«i  "0 


El  [2]  <  (h  .  I)r  .  a 

,r  1  1  2  2  no. 
a//n 


,02 


OR 


yfkl'g[k]  „  K 

j  <  (t,  ♦  _)r  ♦  a 

.  r  k  2'  k  n<\ .  .Ok 

o//n 

(where  a  ,  ....  ......  a  .  ,  converge  stochastically  to  zero  when 

nii  V  n,i  *kk 

a .  is  the  paranetcr  of  the  density  of  X(n)  (i.,...,i,  «  0,1))  wuen 

1 1  1  k  ik 

X(n)  t  Sn (u  ,hj, . . .  ,hk)  ant?  M  =  Kn(X(n)|a0  Q) .  The  a^  ^  . 

1 ‘ ’ ‘  kJ 

(j  =  l,...,k)  ray  depend  on  T(n)  ,y  ,hj , . . .  ,hj, . 

For,  e.p.,  the  first  comparison  of  (5.3.57),  (x | oi0q  q) 

—  *'n^xlu10  o}»  S0  ^  (5.3.55)  and  the  fact  that  <_  x.  (i  *  2,...,k), 


;.B(i)^nj.  _(h  +  i.)r2 

rl  ,  / —  1  1  2  ”l 


l  a’(6)  >_  I  a '  (8)e 
prSk  8eSk 


1  l  a' (8)3 
6eS. 


rl  “ 


12ULL 

of  St\  \ 


1  1  2jrl 


xrum  ,L  u  ? 

ri~ r  -("j*  2)r 

i..  'rt 

From  here  tie  proof  is  essentially  that  which  follows  (5.3.27). 

Rule  for  makin*  comparisons .  For  each  o*  the  k  !  vectors  ij,...,i  , 

one  rust  prove  k  relations  similar  to  (5.3.57),  with  appropriate 


modifications  o^  "<"  to  For  these,  compare  the  pivon  a. 


V^k 


with  the  k  ot' ers  which  have  ij,...,i£'s  which  differ  from  the  riven 
ij,...,ij.  in  only  one  place.  (This  rule,  surpested  uy  the  k  =  ?  results, 


•\rorhs  when  k  >  2.) 
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To  illustrate  our  method,  we  will  now  study,  e.p.,  the  second 
comparison  of  (5.3.57).  Since  Kn(x|anoo  0)  >_  Kn(x|ani0^  Q) , 


r2  J12}  121  -(h2+  \)T\ 

l  a'(8)  >_  l  a'(B)e  o/  n 


BeS, 


8cSk 


X2'Uf2]  K  2 

r2  r  "(h2+  2)r2 
>_  £  a'  (B)e  °An  ] 

BcSk 

B(2)«2 


I  a'  (B) -  [  a '  (B) 


BeS^  BeS^ 


r  ♦  i)r2 

r2  ,  r-  1  2  ?.  2 


1  ♦  _ i(lW _ >_e^/^n 

I  a'(6) 

0eSk 

8(2)-2 

Now  the  proof  proceeds  as  at  (5.3.30),  and  a  relation  like  (5.3.31)  holds 

because  what  is  left  in  £  a' (8)  after  l  a*  (B)  is  removed,  rakes  the 

BeS,  BeS. 

B(2)=2 

"wrono"  associations  and  thus  tends  to  zero,  while  the  denominator  does 
not. 


THEOREM: 

For 

(Xfl],.. 

•»*rk] 

(5.3.58)  (m  ^  j • 

•’P[hl 

'M:  For  0(n*)  and  any  fixed  r  -  (r^,...,^)  >  0, 


possesses,  for  all  r  =  (rj,...,rk)  >  0,  the  pronerty  of 


Theorem  (5.3.13). 
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Proof:  Theorems  (5.3.17)  and  (5.3.52)  establish  conditions  (A')  and 
(B'),  respectively,  for  all  r  >  0.  V'e  therefore  have  a  scnuence  of 
GMLF.'s  possessing  the  property  of  Theorem  (5.3.13),  or  more  properly  its 
extension  to  k  ^  2,  for  all  r  >  0. 

If  T  and  U  are  estimators  of  0,  then  l)  is  said  to  be  more 
concentrated  (about  0)  than  T  if 

(5.3.59)  P.r-r  <  U-0  <  r]  >  Pj-v  <  T-0  <  r] 

for  all  0  e  0  and  all  r  >  0.  (This  definition,  which  appears  for  perhaps 

the  first  time  in  print  in  Lawton  (1968),  is  known  to  the  present  author 

to  have  been  stated  by  Professor  Lionel  '“'eiss  as  early  as  March  1°65  in 

lectures  at  Cornell  University.)  If  T  and  U  estimate  6,  tven  U  is 

J  1  n  n  n 

said  to  be  of  higher  large  sample  concentration  (about  o)  than  T^  if 

(5.3.60)  n0[-r  <  k(n)(Un-0)  <  r]  >  ljm  y-r  <  h(n)  (Tn-0)  I  r] , 

where  k(n)  is  such  that  h(n)(Un-0)  approaches  a  limiting  distribution, for 

all  0  e  0  and  all  r  >  0.  The  GMLE  0^2] ,  •  *  •  »*[>•])  **as»  us^n<1  a 

dimensional  generalization  of  (5.3.60),  desirable  large  sample  concen¬ 
tration  in  comparison  to  the  class  of  estimators  of  Theorer  (5.3.13). 

V.'e  will  now  show  (for  k  =  2,  the  k  >  2  extension  bein'*  similar) 
that,  by  findinn  one  GMLE,  we  find  a  class  of  GMLE ’s . 

LEMMA. :  Suppose  l^m  P0  [Z^  <  y]  =  L(y),  with  L(*)  a  continuous 

n 

(5.3.61)  d'f"  T*16"-  if  ljS  cn  .  0, 

Ijg  ’’g  fz.  <  y*cn'  ■  >  (y). 

n 
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Proof :  If  all  but  a  finite  number  of  the  c  are  positive,  then 

L(y)  <_  l^m  PQ  [Z  <  y+c  ]  and  (since  eventually  all  c  are  less  than  c^, 
n 

m  fixed) 

(5.3.62)  lin  P  [Z  <  y+c  ]  <_  L(y+r  ). 

n  00  n  ^  n  —  ri 

Taking  the  limit  on  m  in  (5.3.62)  and  usin°  the  continuity  of  L(*) 

the  desired  result  follows.  (If  all  but  a  finite  number  oT  the  c  are 

n 

negative,  the  proof  is  similar.) 

If  infinitely  many  are  positive  and  infinitely  many  c^  are 
negative,  s unnose  cr  <  0,cs  >  0.  Then 

(5.3.63)  L(y+cr)  <  l^m  PQ  fZn  <  y+cp]  <_  L(y+cg) 

n 

since  eventually  c  <  cn  !L  cs‘  Taking  limits  in  (5.3.6?)  over 

(r:  cr  <  ^ {s:  >  0}  or  the  £ .  1: .  s .  e"d  r.h.s.  (respectively)  the 

desired  result  follows,  f'otc  that  this  is  a  special  case  of,  with  an 
even  simpler  proof  than,  Cramer's  Theorem  (see,  e.r.,  Pisz  (1063), 
p, 236)  . 


THE0Rr.NI:  If  (7nl  (X(n)  ,r)  ,Zn2(X(n)  ,r) )  is  a  sequence  of  GMLE's 
then  so  is 

(5.3.65)  (7-nl  +  Ojd/fcjOO),  Zn2  +  o2(l/k2(n))}, 

where  o^(l/’  ^(n))  (i  =  1,2)  is  a  quantity  such  that 


o.(l/l.(n)) 
ni“  I?F.  (n) 


lim  V.  (n)o. (1/k. (n)) 

m-°°  iii 


=  0. 


Proof :  I'.’e  will  show  that,  for  the  new  sequence,  conditions  (A1)  and 
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(B’)  (see  (5.3.5)  and  (5.3.6))  hold. 

Since  (A*)  holds  for  the  original  sequence  with  L ( •  1 6)  a 

continuous  d.f.,  it  will  also  hold  for  (5.3.65),  by  Lenna  (5.3.61)  (more 
properly,  by  its  multi- dimensional  analog,  which  is  proven  similarly). 
Since  (B ' )  holds  for  the  oripinal  sequence  {7.^,7^}  with  constants 

anij  =  ^anijl,anij2^  =  it:  wil1  V|°1'*  for  ‘iecIuencft  (5*3-65) 

with  a* . .  given  by 
nij 

an001  =  an00rkl(n)ol(1Al(n,)-  an002  =  an002-k2(n)o2(1/k2tn)> 
an011  “  an0irl:l(n,ol(1/kl(n,)'  °n012  "  an012'k2^°2^Mn^ 

* 

anl01  =  a],1oi-ki(,,)Oj(l/l<[(n)).  anlf,2  *  anl02‘k2‘I’'°2^1/'k2*n" 
anlll  ■  anlirkl(n>°l<1/kl(n))-  a;u2  *  »nll2-k2(n)o2a/k2(n)). 


(Whenever  the  a  .  .  converge  in  probability  to  zero  the  a'.,  do  also.) 
nij  nij 


A  typical  o^(l/k.(n))  might  be  l/(h^  (n)n5i}  with  6^  >  0  fixed 

(i  =  1,2).  In  comparing  any  two  members  of  this  class  of  GMLE's  with 
each  other,  we  find  by  Theorem  (5,3.13)  that  they  have  the  same 
asymptotic  efficiency  (in  the  sense  of  Theorem  (5.3.13)). 

After  results  (5.3.61)  and  (5.3.64)  were  obtained,  the  author’s 
attention  was  called  to  the  latter  part  of  section  3  of  a  preliminary 
version  of  V.'eiss  and  ''olfowitz  (l%7b),  where  a  generalization  of 
Theorem  (5.3.64)  was  stated  without  proof,  hartely,  if  {2^  (X(n)  ,r) , 
Z^C^Cn),!*)}  is  a  sequence  of  GMLE's  then  so  is  { 7-nl+T,!ll »  7n2+Tn2^  w^ere 

(Tnl  ’T^)  *s  suc^  uniformly  in  e. 
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(3.3.66)  Its  PedkjOOTjjl  <  «.  lk2Cn)Tj;2|  <  •  1 

for  any  piven  6  >  0.  Our  proof  can  be  generalized  to  this  case.  (Note 
that  in  the  published  version  of  Weiss  and  V’olfowitz  (1067b)  condition 

(5.3.66)  has  apparently  been  weakened.)  These  results  will  now  be  used 
to  conpare  the  MLE  and  the  GMLE  with  regard  to  asymptotic  efficiency 
when  k  ■  2. 


(5.3.67) 


LEMMA:  For  any  a  >  0,  P  fXr-,-Xr,,  >  a  o//nl  is  minimized 
-  u*  [2]  [1] 

(over  u  e  G(n*)  i.e.  over  y  such  that  y^  3  y^j  +  n  for  some 
n  n*  >  0)  at  yr?|  «  y^  ♦  n*.  Also 
p  ^  .[Xfn-tn  >  a  n//n}>  1  as  n»"°. 

urrurn  n  ^ 


Proof :  By  Theorem  (B.3.2), 


Py  [X[2] _X[1]  >aa/ ^ 


2  2 
.  ifcmJ  _  lfz±IL_] 


a//R>  +e  (o/Gl  fdy 


(5.3.6R) 


•  \  2 

i  "  .  i 

- -  e  dy  +  - 

/Ztt  /FtT 


a 

a - n 

/2o//T 


e  dy 


a—  +n 

_ 

/Fa/  /n 


a 

-a - n 

_/T _ 

/2a/  /n 


1  o 

-  jy2 
c  dy. 


a  ' 

a - n 

_/[ _ 

/2a/*^r 


By  the  formula  for  differentiation  with  respect  to  a  parameter  (e.p., 
Vac's  worth  and  Pryan  (1%0),  n.  2)  or  by  the  Chain  'bile,  since 

(a—  +n)2  >  (a—  -n)2 , 

/n  /n 


o 

a —  >n 
1  /n 


fa—  -n] 


dnPM  fX[2]‘X[l]  >  ao/^nl 


1_  /n  _ 1 _  1  /n 

2  r~  .  r~  m  .  r  2  n.  . 


1  y?.a/A nj  /2o//n  (/2o//nj  /2o//n  > 


Hence  >  a°/^]  is  an  increasing  function  of  n  >n*  >  0,  and 
is  therefore  minimized  when  n  ■  n*  >  9  (i.e.  "hen  u  =  ufl]+n*^‘  ^’at 
this  minimal  probability  >1  as  follows  fro^  (5. 3. *8). 


(5.3.69) 


LFfW  : 


(5.1. 39)  . 


1 - e  <2,  where  e  is  the  nositive  solution  of 

,2  0  —  O 


Proof :  From  (5.1.39)  and  the  fact  t'  at  coth  (x)  >  1  for  x  >  o. 


hf"  'Go  =  VcoCOth(eo/2)  =  f0(coth(co/2)-l). 

1  o  ' 

Usin'’  an  exoression  for  coth(e  /2)  which  was  found  in  the  proof  of 

a  1 


Lenina  (5.1.33),  this  becores 


c  /?  -c  /2  -c  /2 

j2_  o'  o  -  o  e 

.  e  =  c  - - — - 1  -  e  — - - =  "> — - —  <  2 

02  o  o  Eq/2  -e  / 2  o  e  /?  -e  /2  "  eQ  -  * 

e  -e  e  -e  e  -1 


XX  v 

since  (for  x  >  0)  x/(e  -1)  <  1,  or  x  <  e  -1,  because  x+1  <  e' 


+  . . . . 
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In  the  notation  at  (5.3.66),  we  wish  to  show  that  the  MLF 
{Xfl^Tnl’Xfz]^}  is  such  that  (5.3.66)  holds,  with  k^(n)  =  k2(n)  = 

*^n/o.  By  Theorem  (5.1.37), 

lTnl  I  =  ^[l]'Xfl]  I 


(5.3.70) 


xm*xm  ^ 

?  '  xm 


X[1]"X[2]  X  f  2 ]~X[ 1 ]  - 

2  "  2coWfrj7)-1\l} 


x[2i'xm 

2 


Xf2] ~Xfl] 


1- 


1 


COth  (e  72 y 


if  0  <  X^-X^j  <  /?o//iT 


if  Xf2]'y[ll  > 


if  n  1  X[?]'Xfl]  - 


if  vf2i ~xrn  >  /*al^ 


and  |T^2I  s  |u  r?i~xm  I  turns  out  to  he  the  same.  Thus,  usinp  the 


r2]  ~[2] 


definition  d  =  and  the  fact  that  eQcoth(eo/2)  =  d2n/o2,  for 


any  6  >  0 


lylklOOT'il  <  6.  !k2(n)Tn2l  <  61  =  PurfTni!  <  &a/^ 

P  rx^T-X,.,  <  26 a//rT,  0  <  Xr_,-Xri1  <  /fo/.'nl 
M  [2]  [1]  -  f/|  [ 1 1  — 


+  P 


x[2Txm 


1_  coTHTTT^) 


<  to/Sn,  >  fio// n 


>  P 


xm:xm 

2 


1_  cotb  (z/T) 


<  6o//n,  X|.2j-Xj.jj  >  /2o//n 


(5.3.71) 


106 


V  _  Y 

‘  \2]  xri] 


C  O2 

1-  -2-- 
d2n 

<  5/^  xm’xm >  ^ o/ ^ 

x[2fxm  i^h-v2 


d2n 


<  6o//n,  > 


.  |d2n-c  o2 |  _  _  _ 

2  T2  <  (X[2]'X[l])fo/l/"*  X[2]"X[1]  >  ^ 0/ ^ 


Y  -Y*  >  i _ £ 

[2]  X[l]  25 


& 


d2n 

- 1 

2  0 


XT2]  "X f  1]  > 


THEOREM:  For  the  MLE  when  k  =  2,  uniformly  in  g  ,  for  any 
(5.J.72)  8lven  5  >  °- 

AiE  pu[lki(n)TAil  ‘  ■  '• 


Proof :  By  Lemma  (5.3.69)  and  equation  (5.3.71), 

Pg[|kl(n)Tnl!  <  5'  lk2(n)T^2l  "  61 

(5.3.73)  L  Pu  fX[2]'X[l]  >  15  ^2’  X[2l'Xri]  >  / ^ 


S  PMfV[2)'Xri)  *  pax(*/^,j)o//nl . 

By  Lemma  (5.3.67),  the  last  member  of  (5.3.73)  can  be  bounded  below, 
for  g  e  0(n*),  in  such  a  way  that  the  bound  -*■!  as  n>®. 


By  Theorem  (5.3.72)  it  follows,  as  noted  above  (5.3,66),  that  the 
MLE  and  the  G!!LE  have  (for  k  =  2)  the  same  asymptotic  efficiency,  and 
that  the  MLE  is  a  GMLE.  This  proves  asymptotic  efficiency  properties 

for  the  MLE  which  do  not  follow  directly  from  the  standard  theory,  which 
assures  i.i.d.  observations. 


CHAPTFR  5.  POINT  ESTIMATION:  MAXIMUM  LIKE LI HOOD  (ML) 
AND  RELATED  ESTIMATORS 
5.4.  MAXIMUM  PPOPARILITY  ESTIMATORS  (MPE's) 


Maximum  probability  estimators  were  introduced  Ky  We iss  and  Wolfo¬ 
witz  (1967b)  for  much  the  same  reason  as  GMLF.'s  were  introduced  by  Weiss 
and  Wolfowitz  (1966),  as  discussed  in  Section  5.3  above.  Weiss  and 
Wolfowitz  (1967b),  pp.  202-203,  proved  t^at,  for  the  case  of  n  «  1 
parameter,  every  GMLE  is  an  MPF.;  thus  MPE's  extend  the  notion  of  CMLE's 
(and  by  finding  a  GMLE  we  find  a  fortiori  an  MPF) .  I'e  now  study  the 
extension  of  this  result  to  r  >_  1  parameters,  first  summarizing  Weiss 
and  Wolfowitz 's  results. 

Let  G  and  0  be  as  in  (the  r-dimensional  analop  of)  (5.3.1),  let 
X(n)  be  as  in  (5.3.2),  and  let  (x | 0)  and  u„  be  as  in  (5.3.3). 

DEFINITION:  Let  R  be  a  repion  of  if ,  let  k(n)  = 

(k j (n)  , . . .  ,k^(n))  be  such  that  k(n)+®,  let  d  *  (d^ , . . .  .d^) , 
and  define 

d  -  R/k (n)  =  {(z1,...,zp)  c  0  :  di  -  y./k^n)  =>  z i, 

i  =  l,...,m,  (yx , . . . ,yn)  e  R). 

DEFINITION:  is  a  maximum  probability  estimator  with 

respect  to  R  and  k(n)  if  (for  a.e.(^n)  value  x  of  X(n)) 

Zp (x)  equals  a  d  e  G  such  that 

I  ...  j  Kr(x|e)de  ...d9  ■  sun  /  ...  /  T-  (x|e)d6  ...de  . 

d-[k(n)]_,R  teG  t-[k(n)pR 


(*>•4.1) 


(5.4.2) 
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CONDITION:  Por  each  li  >  0  and  0q  c  0 

(5.4.3)  lim  P,  [!;(n)(Z  -0)  c  Pi  »  8 

fr>  co  0  J* 

uniformly  for  all  0  c  If  »  (0:  |k(n)(0-0Q)|  £  h). 

CONDITION:  For  each  0  c  0 
- -  o 

(5.4.4)  few  Pe[|k(n){Zn-0)|  <  M]  -  1 
uniformly  for  all  0  in  some  neighborhood  of  0  . 

CONDITIO!':  Por  each  0  c  0  and  h  >  0 
-  o 

(5.4.5)  few  {P0[k(n)(Tn-0)  c  P]  -  PQ  fb  (n)  (T^)  c  R] }  =  0 

uniformly  for  all  0  e  H  *  {0:  |V.(n)(0-B  )|  h). 

TPEQRHfi:  Let  { Z^}  be  an  MPE  with  respect  to  P  and  k(n). 
Suppose  { 7  }  satisfies  (5.^.3)  and  (5.4.4).  Let  {Tn>  be  any 

(5  4  6) 

estimator  wide1,  satisfies  (5.4.5).  Then  (for  each  6  e  0) 

s>.  C  ("HTn-V  ‘  Rl. 

O 

TUFOPEM:  Let  l\fn  be  a  GMI.P  (with  respect  to  r  =  (r^,...,^) 
>0)  for  the  estimation  of  0  =  (0j,...,0^)  e  G  (m  >_  1) . 

Choose  R  =  { (yx , . . .  ,yp)  :  -r./7  <  y.  <  r./2,  i  =  l,...,n) 
(5.4.7)  and  k(n)  as  for  the  GMI.P.  If  the  MPF  (w.r.t.  this  P.  and 

k(n))  satisfies  (5.4.3)  and  (5.4.4),  and  if  the  CMLF  satisfies 

(5.4.5),  then  the  GMLF  is  (in  the  equivalence  class  oC)  such 
an  MPF. . 
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Proof:  Let  Z  be  the  MPE  w.r.t.  this  R  and  k(n).  It  then  satisfies 
-  n 

the  condition  of  Theorem  (5.3.13).  Thus  (for  each  0q  e  0) 

(5.4.8)  yBP8  (kCn)(Wn-eo)  c  n]»_psPe  [k(n)(Zn-V  c  • 

0  0 

The  GMLE  Wn  satisfies  (5.4.5)  and  thus  the  conclusion  of  Theorem  (5.4.6) 

holds :  for  each  9  c  0 
o 

(5.4.9)  ympe  fk(n) (2  -e  )  c  R]  >_  Dg  P,  [k(n)(W  -e  )  £  «]. 

o  0 

Then  (see  Weiss  and  Wolfowitz  (1967b),  p.  198)  the  GMLP  is  (in  the 
equivalence  class  of  such)  an  MPE. 

The  result  of  Weiss  and  Wolfowitz  (l%7b)  for  the  case  m  *  1  is 
somewhr  t  stronger  than  our  Theorem  (5.4.7)  for  the  case  m  >_  1 :  they 
show  that  the  MPE  satisfies  (5.4.3)  and  (5.4.4).  (They  assume,  as  we 
do,  that  the  GMLP.  satisfies  (5.4.5),  which  is  stronger  than  (A')  of 
(5.3.5).)  Our  result  (more  precisely,  a  slight  extension  of  our  result) 
says  that  if  the  MPE  for  a  problem  is  "good"  (i.e.,  satisfies  (5.4.3) 
and  (5.4.4)),  then  the  GMLP  (if  it  meets  (5.4.5))  is  equivalent  to  it. 
Note  that  the  analog  for  it  >  1  of  Weiss  and  Wolfowitz *s  result  for  m  =  1 
is  false.  E.g.,  Weiss  and  Wolfowitz  (1967b),  p.  ir>g>  iast  paragraph, 
note  an  example  (with  n  =  2)  where  the  MPE  is  not  "good"  although  the 
GMLE  is.  (Weiss  and  Wolfowitz  give  a  method  for  attacking  the  problem, 
in  such  cases,  by  modifying  it  slightly  and  thereby  obtaining  (often 
"good")  MPE's.) 

We  will  now  study  in  detail  the  MPE  of  the  ranked  means.  Although 
we  have  seen  that,  in  general,  for  n  >  1  parameters  even  if  a  GMLE  and 
an  MPE  both  exist  the  MPE  may  not  be  pood,  in  our  case  the  MPE  is  shown 
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(for  the  case  i>  ■  2)  to  have  all  the  pood  properties  of  the  fiMLF.  Thus, 
let  0  ■  (p  p  c  p  j .  ,u  j }  and  0  ■  R*  ,  an-!  let  *(n), 
Kn(x|p),  Mn  be  as  specified  in  (5.3.14).  Fix  r  ■  (Tj,...,^)  >  n,  and 

choose  kj(n)  *  ...  =  kk(n)  *  'Vo ,  n  =  { (Vj . y,.):  -r./?  <  ^  r^/2, 

i  *  1 , . . .  ,k } .  Then 

1. 


d  -  [k (r)  ]  R  =  { (z1,...,zk)  t  0: 


(5.4.10) 


d.  -  y./V.(n)  =  z.,  i  =  1 . k ,  (y. . y.)  g  R} 


and 


r.  r. 

{(Zj.-.-.z,)'  d.  -  -2y-jny  I  zi  <  ^  *  2TTTnT’  1  8  1  •  ■  ■  ■  • 

i  l 


sup  /...  /  Kn(x|p)dp^...du^ 
tc0  t- [k (n)  1  1 R 


(5.4.11) 


r,.  r 

t^  ♦  t ^  ♦  j^o//n 


sup  t  /  ...  /  (x  |p  )dp  r  i  j  •  •  •  dp  fi,  i 


t  1  t 

V*,,,lk  r 


tk  ‘  y°Vn 

For  the  case  >.  =  2,  (5.4.11)  becomes  (when  *  x^  and  =  x7) 


t2  ♦  ^  o//n  tj  +  ^-o//n 

su’i  _n _  /  / 

W  2tto2  r  r 

t7  -  2~o//n  tj  -  — o//n 


’ 

1 

rxrp  rn] 

2 

i 

r*?^  ml 

"2 

e 

.  o//n 

~2 

o/v^n 

+  e 


111 11 


o/ /n 


x,-p 


1  f  2 1 


{  o/^n  J 


dufi]<kJr2] 


Ill 


(5.4.12) 


(5.4.13) 


sup 

V*2 


V^2  T_2 

o/4T  2 


t2*X2  r2 


j/i^n 


Vxi  ^1 

o/^  2 

f 

#  J  m 

ta 

a//Z  2 


1  2  1 
i  ’  in  i  y 


/27 


2 

2V2 

e  j  dv  j 


t2’xl  r2 

a//S  2 

*  I 

t2~xl  r2 


trx2  ri 

2 


1  2 


o/«^n 

f  J..' 

J  /2tT  /2tt 


*rx2  ri 


j/i/n 


1  2 
"  2V2 

e  dvj  dvp 


sup 

tl’t2 


VX1  +  "l 

2 


[o/*^ 


-  0 


VX1  ri 


[o//n 


’ 

f  > 

t2_X2  r2 

fVx2  r2' 

« 

4> 

U  *  rl 

-  $ 

Q 

5i 

i 

N>| 

- 

♦  < 


• 

♦ 

trx2  +  ][i 

-  4 

trx2  ri| 

o/Jn  2 

2  J. 

VX1  T_2 

2 


o/J n 


-  t 


VX1 


j/*^n 


;  /j 


LEffJA:  Let  d  =  ^(x2"xi^»  tl  ’  xl  +  aia/*^"»  t2  “  x2  *  &2a^‘ 
Then  an  MPE  is  (tj,t2)  with  aj,a2  which  achieve 

sup  [{♦(a1+r1/2)  -  ♦(a1-r1/2) }{$(a2+r2/2)  -  $(a2-r2/2)} 

Va2 

♦  {$(a1-d+r1/2)  -  $(aj-d-rj/2)  }{<S>(a2-d+r2/2) 

-  4>(a2-d-r2/2)}] . 


Proof:  By  definition  (5.4.2),  for  our  case  as  specified  above  (5.4.10), 
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the  MPE  is  which  achieves  the  sunremun  in  (5.4.12).  If  we  use 

d  =  — (x2-Xj)  anrl  transform  via  tj  =  x^  +  a^o/Zii,  t2  =  ^  -  a^o/^n,  this 
(t l * 1 2^  wiH  be  specified  by  the  (a^.a^)  which  achieves  the 
sup  [{$(a1+r1/2)  -  l>(a1-r1/2) } { <h (-a7+r2/2)  -  <J>(-a2-r2/2) } 

Va2 

+  (<I>(a1-d+r1/?)  -  <J(a1-d-r1/2)  }{$(-a2+d+r2/2)  -  $(-a2+d-r2/2) }] . 

IJsinp  the  relation  <J>(x)  =  1  -  4(-x)  (x  e  f?) ,  this  becomes  as  specified 
in  the  statement  of  the  lemma. 

LEMMA:  The  sunremum  o^  (5.4.13)  occurs  only  at  (a,, a-,)  with 
(5.4.14)  1  “ 

0  <  a^  <  d,  0  <  a2  <  d . 

Proof:  By  reasoning  as  at  (5.1.5),  the  simrenur  must  occur  nt  a 
critical  point.  However,  if  we  sot  the  partial  derivative  with  rcspect 
to  a1  equal  to  zero  we  obtain 

<Ka!+ri/?)  -  <J> (a^-rj/2)  4>(a.,-d«-r2/2)  -  $(a?-d-r7/2) 

iKaj-d+rj/2)  -  ^(aj-d-r'j/T)"  '  <j>(a.,+r2/2)  -  <k(a^-T^/?)  . 

Since  the  r.h.s.  is  always  <  0,  the  l.h.s.  must  always  he  <  0.  How,  the 
denominator  of  the  l.h.s.  is  positive  (negative)  ifr  <  d  (a^  >  d). 
Thus,  we  must  have 

r  r.  <  0  if  a.  <  d 

♦  (a.+  y~)  -  *(8.-  -~)  1 

>  0  if  a j  >  d 

i.e.  aj  >  0  if  aj  <  d 

aj  <  0  if  aj  >  <?• 

This  proves  the  result  for  a^:  the  result  for  a2  follows  similarly. 
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LFMMA:  By  imposing  a  consistency  criterion  for  estinators 
(5.4.15)  similar  to  (5.1.4),  we  may  restrict  ourselves  to  (a^a,)  with 

aj  +  a2  i 

Proof :  In  order  that  we  have  t^  <  t,,,  we  must  have  x^  + 

/r 

<  x2  -a2o//n,  i.e.,  +  a?  _<  -(x^Xj)  =  d- 

Note  that,  in  the  region  of  (al5a2)-space  in  v/hich  Lemma  (5.4.14) 

tells  us  the  su^remum  of  (5.4.13)  must  lie,  we  have  symmetry  (oh  values 
of  (5.4.13))  about  the  line  a^  +  a0  =  d;  see  Figure  (5.4.16).  Thus,  our 

consistency  criterion  only  eliminates  -n  illopical  dunlicato  f-axirizinw 
point . 


a2 


Figure  (5.4.16) 


LPMMA :  For  any  fixed  6  >  0,  there  is  a  Kfr^r^fi)  such  that 
if  d  >  K(r. ,r_,6)  then  (5.4.13)  is  maximized  (ir.  the  shaded 

(5.4.17) 

region  I:  a^  >  0,  a2  >  0,  +  a2  <  d  of  Fipure  (5.4.16)) 

inside  the  disk  0:  a^  +  a^  <  6. 

Proof:  Let  fj  =  (4>(a1+r1/2)  -  Haj-rj/2)  )(<J>(a?+r2/?)  -  <t>  (a2-r7/2) } , 

f2  =  (<I'(a1-d+r1/2)  -  <J>(a^-d-rj/2)  }{^(a2~d+r2/2)  -  0 (a2-d-r2/2) } ;  then 

(5.4.15)  is  sup  (f  +f  ). 

(a1,a?)in  I 

ilow  over  (a^a^,)  e  I,  ?j  is  maximized  at  (a^,a2)  =  (0,0)  am’ 

decreases  as  a^  and  a2  increase.  Tims,  if  we  rove  (a^.a^)  outside  P, 

the  loss  i.  is  at  least  f j  ((0,0))  minus  the  largest  value  of 

f^((aj  ,..9))  on  the  boundary  of  0  inside  I;  there  a^  +  a^  =  6,  so 

sup  f  ((a  ,a  ))  =  sup  {♦(a1+r./2)  -  $(a. -r./2) } • 
a*4a‘  =  6  1  1  '•  Ofa^fi  11  11 

(a!  .'n2)  in  I 

•  {$ (/i^a^+r2/2)  -  a(/6- a^-r.,/2)  ) 

<  {<t>(Cj6+r j/7)  -  4>(c16-r1/2)  }{<I>(r2/2)  -  * (-r^/2)>, 

where  we  nay  suppose  without  loss  that  c^  =  Cj(rj,r2,<5)  >  0.  (This  can 

only  fail  if  the  sunrenum  occurs  at  (a^,a2)  =  (0,6),  in  which,  case  we  may 

reverse  the  roles  played  by  a^  and  in  our  inequality  and  the 
argument  below  will  ro  through  similarly.)  Tims,  the  loss  in  fj  via 
yoinn  outside  P  is  at  least 
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{♦(rj/2)  -  4>(-r1/2)H*(r2/2)  -  ♦(-r2/2)> 

-{♦(Cjfi+rj/2)  -  «(c15-r1/2)H4»(r2/2)  -  -f(-r2/2)> 

»  {♦(r2/2)  -  *(-r2/2))[{*(r1/2)  -  *(-r1/2JH*(c16+r1/2)  -  ♦(c1«-r1/2)}l 
■  c2(r2)c3(r1,r2,6)  (say). 

The  gain  in  f2  (which  increases  as  and  a?  increase  in  T)  is  less  than 

sun  4>(a1-d+r1/2)'J>(a7-d+r,/2) 

(a1,a?)  in  I 

sup  <?(a.  -d+max(r.  ,r,))<J>(a0-d+max(r1  ,r?)) 

(ai,a2)  in  I  1  1  1  ' 

=  sup  ^(a. -d+max(r.  ,r, ,-d+rax(r.  ,r_)) 
a1+a2=d  1  it-  i 

ara2>P 


*  sup  $  (aj-d+r>ax(r1,r2))i>(-a1+inax(r1,r2)) . 

Olajld 

We  will  show  that 

fc  A  lin  sup  «t(a  -d+max(r.,r  )  )<5>  (-a. +r.ax(r.  ,r.) )  =  0, 

t5-4-18'  0<a  <d  1  17  1  12 


Thus,  there  will  exist  a  K(rj,r2,6)  such  that  d  >_  K(rj,r2,6)  implies  the 
gain  is  less  than  c2(r2)  c3(r^,r2,6),  which  will  prove  the  lenra. 

Let  X  and  Y  be  i.i.d.  N(0,1)  r  v.'s.  Then  (5,4.18)  is  equal  to 

rt.  .  1Q,  lin  sup  PfX  <  a  -d+iax(r1,r  ),  v  <_  -a.+naxfr.  ,r-)] , 

d*«  0<aj<_d  1  1  ^  1  1 


which  involves  t^e  probability  in  a  certain  rectangle  in  R2,  as 
illustrated,  in  Figure  (5.4.20). 


(maxCrj.r 


Figure  (5.4.20) 
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Thus,  (5.4.19)  is  less  than  or  equal  to  the  limit  of  the  supremum  of 
the  probability  to  the  left  of  the  line  X  +  Y  =  -d  +  max(rj,r2), 

lim  sun  P[X+Y  <_  -d+nax(r.  ,r_) ]  »  lir  P[X+Y  <_  -d+max(r.  ,r«)]  ■  0. 
d+°°  0<aj<d  d+  °° 


THEOREM:  For  y  c  0(n*)  (see  (5.3.15)),  the  MPE  (tj.tg)  is 

(5.4.21)  equivalent  to  the  GMLE  (X^  .X^)  found  in  Section  5.3,  and 

thus  has  the  sane  ontinun  nrooerty  as  that  CMLE. 

Proof :  lie  wish  to  show  that,  for  each  \i  e  0(n*)  and  for  each  fixed 

5  >  0, 

1  =  lim  P^ (k(n)max(|t1-Xj1j | , |t2-X^2j |)  <  6] 

=  lim  P  [— rnaxdaj  lo/t^n,  ja2|a/*/n)  <  6] 
rw°°  p  0 

=  lim  P  [max(a.,a2)  <  6] , 

IT*®  M 


where  the  last  equality  uses  Lemma  (5.4.14).  Now  by  Theorem  (R.3.2), 

Xn-  - 

the  density  of  d  =  [2]  ^[1]^  *?or  X  —  0  *s 


1 

2  /tt" 


y- 


o/i/n) 


+  e 


y+ 


o/'fn \ 


where  n  =  y  ^  -  u  Thus  lim  fd  >_  Kfr^  ,r0,<5)]  =  1,  so  usine  Lemma 


it*® 


(B.2.1), 


lim  P  fmax(a.,n.2)  <  5]  =  lim  p  (>ax(aj,a2)  <6  |  d  >  K(rj,r2,5)]  =  1, 
n*®  u  m®  P 

where  the  last  step  uses  Lemma  (5.4.17). 


A 


CHApTFp  6.  TMTFPVAL  FSTIMATION 


6.1.  nr:::rv>L  f^rptation 


Consider  joint  confidence  interval  estiration  of  »*  * 

Dur  observed  statistics  under  ’tile  (1.3.2)  are  X^  (i  =  l,...,k* 
j  =  l,...,n)'  vc  take  to  be  fundamental  as  at  (5 .1.1)  (note, 

as  has  been  pointed  out  by  Bechhofer,  Kiefer,  ami  Sobel  (ln68),  Part  I, 

Remark  4.1.2,  that  Xj . X^  are  sufficient  and  transitive  for  g  ^ , . . .  ,Uj. 

after  n  stages;  see  p.  4.?6  and  Theorem  ln .  1  of  Bahadur  (1054),  as  veil 
as  pp.  334ff  of  Ferpuson  (1067)  for  details  of  these  notions),  choose 
our  interval  to  be  of  the  form 


I  =  KV-’V 


(6.1.1) 


=  (f  fjj » •  •  •  u  p.] ;  "j  I  R  m  .1  bi»  •  •  *  M  r?0  -  \ ^ ’ 


where  pj  ,h  ^  ; . . . ;  are  functions  of  and  ask  tv/o  invariance 

conditions  (involvin'-  relabeling  of  populations  and  shifts  of  location). 


(6.1.2) 


rY?!MFTnY  IMVAnTAMCF ■  For  all  B  t  ^ 


I(X1,...,Kj.)  =  T(X|,(1),...,y6(,.)). 


(6.1.3) 


LOCATION  IMVAPIAPrn:  For  all  c  e  R, 

I  (Xj+c , . . .  ,Xj  +c)  =  I  (Xj , . . .  ,7^)  +  c. 


h'eiss  (1063)  pointed  out,  in  another  context,  that  (6.1.2)  and  (6.1.3) 
are  not  necessarily  the  only  or  the  best  ways  to  compensate  for  nermuta- 
tions  and  shifts  of  location,  respectively:  there  may  be  other  ways  to 
compensate  which  yield  the  same  interval. 
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LEMMA:  Under  condition  (6.1.2),  I (X, , . . . ,yL)  must  be  of  the 

(6.1.4) 

form  I (Xjji ..•• »y|k]) ’ 

Proof :  Condition  (6.1.2)  implies  that  I  depends  only  on  the  ordered 
X.  (i  -  1, . . . ,k)  . 

DEFINITION:  Let  a^. . .  ,ak  (ax  >  0,  ...  .a^  >_  0-a1  ...  ♦  a^ 

(6.1.5)  =  1) ,  b*  (0  <  b*  <  ®) ,  and  (G , I ')  (-»  <_  G  H  <_  +00)  be  con¬ 
stants  pre-set  by  the  experimenter. 


'Ve  now  take  our  loss  function  to  be  a  weighted  sum  of  the  proba¬ 
bility  that  I(Xjjj , . . .  ,Xjkj)  doesn't  cover  Wj-.j  plus  a  multinle  of  a 
quantity  related  to  the  length  of  the  interval  on  (i  =  l,...,k): 

LOSS  FUNCTION:  W(u  ; I  (X ^  , . . .  .X^j))  = 


(6.1.6) 


■  l  a.  (P fum  t  (E-,h  )]  +  b*  min(h.-g.  ,N-C)J. 
i*l  l  u  H  !  ii  ii 


Note  that  the  length  lu-g^  is  the  special  case  of  min (h^-c^ ,H-G)  where 
the  experimenter  chooses  (G,H)  with  H-G  =  +®. 

RISK  FUNCTION: 


(6.1.7) 


r(y  ,I(Xjjj)...,Xjkj)) 


E  W(y 
U 


I(X 


[1] 


))• 


Thus, 


r(u;I)  =  F^Orl) 


•  iI1aiEu{Pp6l[i]  *  *  b*  min(h.-c.,M-C)} 


(6.1.8) 
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i  ai  rll  fi  l  *  ,!•>.)]  ♦  b*  l  a.E Biinfh.-p  ..H-H) 

i=l  1  M  llJ  11  i-1  1  w 


=  1  - 


U^{i]  c  (p.,h.)]  ♦  I^a^minfhj-Pj.H-C). 


Our  aim  now  is  to  find  functions  ,hj ; . . .  .'R](»hj(.  which  are  in  so*"e 
sense  optimal  with,  respect  to  (6.1.7),  o.o.  which  achieve  the  minimum 


(6.1.0) 


inf  sun  r(u  ;I(Xfn  , . . .  ,Xf, .,)) 

..;ok,h,.  ueO(furv1) 


and  provide  a  minimax  invariant  confidence  interval.  (The  a  in  (6.1.9) 
will  be  non -randomized,  since  p  is  a  fixed  anlnown  and  not  a  random 
variable;  I(X^j , . . .  will  be  considered  non-randomi  zed  also.) 

Although  we  have  been  unable  to  carry  out  (6.1.9)  or  other  ootimization 
in  the  general  case,  results  for  special  cases  are  obtained  below.  Mote, 
for  use  below,  t’  at  by  Lemma  (6.1.4)  and  (6.1.7)  with  c  =  -X^j  v'e  have 


TMEOP.CM :  Under  conditions  (6.1.2)  and  (6.1.3),  I (X^ , . . . , X^) 
must  be  of  the  form  -  (v  j-j  ]  >  •  •  •  >*  p.-j )  with  (for  i  =  1,. ..,’;) 

*i  =  X[i]  ’  f7i(Xfi]'X[l] . X|i]'Xr»:)) 

hi  =  "[i]  +  hi(X[i]’Xri],,**’Xri]"Xrk])‘ 


(6.1.10) 


i 


CHAPTER  6.  INTnnVAL  ESTIMATION 
6.2.  INTERVALS  OF  FIXED  ’"IDTH  WITHIN  A  CERTAIN  S"BCLASS 


In  Theorem  (6.1.10)  we  looked  at  the  form  of  intervals  of  type 

(6.1.1)  under  two  invariance  conditions.  We  now  study  the  subclass  of 
joint  intervals 

IN(*[l],,,,’Vl)  "  ri]  ’ ' ' '  *U  fk] : 

(6.2.1) 

Xfl]'gl  —  u  [1]  -  X[l]+hl’**”X[k]‘pk  -u[k)  -X[k]+hk}* 
which  utilize  the  "natural"  estimators  of  (i  ■  l,...,k) 

strongly  by  taking  gj,h*  ; . . . ;g£,h*  to  be  constants.  Further,  we  will 

suppose  the  experimenter  has  specified  nositive  constants  dj,...,d^,  and 

wishes  the  interval  about  y  ^  to  be  of  lenpth  (i  =  1,...,1<).  We 

then  study  intervals  of  fixed  width  within  subclass  (6.2.1),  i.e.  the 
subclass  of  joint  intervals 

IF,:,'^[1]  *’  ’  ’  ,Xfk]^ 

(6.2.2)  =  X^j  +  fhj'-dj)  f-Hj-jj  J » •  ••  i 

*[k]  +  (hk_<V  —  u  [k]  -X[k]+hk}‘ 

Then  (here  it  is  logical  to  choose  (G,H)  =  (-«,+“>)) 


r(‘,:IF.N) 


Ik 

(6.2.3)  =  1  “  JjVu^ri]  e  (-[i]  +  (hi-di^X(i]+hpl  +  b*i?1aidi 

k  k 

1 1  *  ■  J, ‘ili [‘,nrfc!  -•  hi]  i-fil-M-ii. 


1 21 


k 


which  is  of  the  form  constant  (specified  by  the  experimenter)  minus  a 
weighted  sum  of  probabilities  of  coverage  of  u  ^*nd 

h*,...,h£  which  are  optimal  in  the  sense  of  (6.1.9)  (minimax)  within 
subclass  Ip  N  of  (6.2.2),  we  must  find  the  hVs  which  achieve 

k 

(6.2.4)  sup  inf  l  a.P  fyr..-h*  <  Xr.,  <  p  r  •  i . ] . 

hj,...,h*  McnoCi[k])  i.1  1  wl  [1]  1-  [1]  -  M  1  1J 

For  the  case  aj  «  ...  «  a^  .  »  0,  a^  »  1,  suppose  we  set  h*  °  d^/2 . 
Then  Lai  Saxena  and  Tong  (1968)  claim  in  an  abstract  that 

inf  VP[k]"dk/2  -*rk]  -u  fk]+dk/21 
yeVu[  k]5 


k  /n 

occurs  atviri,  =  ...  =  y  r,  , ,  and  therefore  equals  4>  x~  — 
[lj  l x J  .  U  oj 


4>^-  ^ - -Jj  ;  i.e.,  if  one  uses  the  interval  (X^^-d^/2,  X^^+d^/2) 


for  w  r,  ,  then  the  probability  of  converge  is  a  minimum  when 


[1]  =  =  w[k]- 


D1APTFP  6.  INTERVAL  ESTIMATION 


6.3.  I'PPF.P  AMO  LO'TP  INTFrVALS  WITHIN  A  CERTAIN  SUBCLASS 


The  subclass  of  joint  intervals  1^  of  (6.2.1)  utilizes  X^j  as  an 
estimator  ofp^j  strongly  (i  =  l,...,k).  For  problems  in  which  we 
wish  an  upper  (lower)  joint  confidence  interval  on  u  ^  , . . . >u  ^  we  will 
set  gj  =  . . .  =  g£  *  +®  (hj  =  ...  =  h*  =  ■*■•»)  in  (6.2.1).  Then  our 
interval  is  in  one  of  the  classes 

(6.3. 1)  yt,  =  M[j]  I  '[l]+hl  ’  *  *  * fk]  -  "fk]+bk} 

(6.3.2)  IM>L  =  {u  [  1} '  *  *  *  *P  [k]  :  ^(1] ~  p  [1  ] '  *  *  *  *^fV]  nk  -u[k]} 


(6.3.3) 


r(u:IN>l,)  -  1  -  JiaiPurX[.]  iMfjj-h}]  +b*(H-G) 


(6.3.4) 


r(u  ,1 


r!>L)  =  1  ’  Jj'iV^fil  -  u  fi7+Pi3  +  b*(H_n)- 


For  the  case  of  uaper  intervals  we  may  choose  H-G  =  0  without  loss. 


(6.3.5) 


r(y:IH,U)  “  J1aiPpfXfi)  -VJ(i]"hi1- 


Similarly,  for  the  case  of  lower  intervals  we  may  choose  H-G  =  0 
without  loss.  Then 


(6.3.6) 
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THEOREM:  For  any  i  (1  <  i  <  k),  if  a,  *  1  (thus  a^  *  0  for 

j  j*  i)  then  the  risk  (6.3.5)  ((6.3.6))  is  the  probability 
that  our  upper  (lower)  interval  doesn't  cover  p^j,  and  is 


(6.3.7) 


i-1  terms  k-i+1  terms 
maximized  over  p  e  ^(p^)  at  p=  (-»,...  ,-®,p^^  , ...  ,p  j^) 

i  terms  k-i^terms 

(p  =  (Pjij,...,Pjij 


Thus,  for  any  y  (0  <  y  <  1)  an  upper  (lower)  confidence 

interval  of  minimal  probability  of  coverage  y  is  (-®,X,.,+h?) 

L  ^  J  * 


((X[i]-g?,+«))  with  h?=(o//n)<t‘1(Yk"1+1)  (gt  =  (o/Zn)#"1^1))  • 
Proof :  Upper  Interval.  For  any  i  (1  _<  i  _<  k) ,  if  a^  =  1,  a^  =  0  (j  i  i) 


1 

-1,  U 


sup  r(p;I  ) 

P^ti]’ 


sup  pu  txrn  — w  riThi3  =  sup  Fx  Cm  ril  "bp 

:Qo(u[i]5  Pep,>r4i)  fil 


oVM  [i] 


*  lim  P  [Xr •  -i  <pr;1-hj] 

}■*>+»  y  [l]"*"^  [i-l]  =  "M,Mfi]  =  ",sv  [k]  [l]  1 

since,  for  i  =  l,...,k  and  x  e  R,  F7  (x)  +  as  p.t  (£.  =  l,...,k)  by 

X[i]  * 

Theorem  (2.1.11).  It  follows  by  a  modification  of  the  proof  of  Case  1 
of  Theorem  (2.2.4)  (using  1  for  x)  that 
sup  r(u;INU)«P 

P£fio(p  [i}5  ’  U[l]=  '^[i-l)"’00^  [i)  =  *  • 


=  P  min(Y  )  <  -- 

1  k*1'  o//n 
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where  Yj , . . . »Yk_^+1  are  k-i+1  independent  N(0,1)  r.v.'s.  To  make  the 
minimal  probability  of  coverage  y  (0  <  y  <  1)  we  set  h*  so  that 


1  -  y  ■  P 


min(Y1,...,Yki+1)  1~~ 

o//n 


1  -  P 


-hi 


min(Y1,...,Yk_i+1)  >  — ^ 


=  1  - 


p 

f-h? 

l-$ 

X 

c//nj 

k-i+1 


h* 

x 


(o//nj 


k-i+1 


f  h* 

k-i+1  1 

h* 

thus  y  * 

$ 

l 

°//n. 

k-i+1 
,  Y  s 

i 

p/^n. 

,  and  ht 


(o//n)<t"1(Yk”i+1) 


Lower  Interval.  For  any  i  (1  <  i  <  k),  if  a.  *  1,  a.  =  0  (j  ^  i) 

X  j 

then  by  Theorem  (2.1.11) 

sup  rfii;IN  ,)  -  sup  p  [Xm  >um+gt] 

11  ‘] 


■  lim  P  ,.fXr.,  >  V  r-i+P*l  • 

w-  *,[u'—"uiiru(i»i]*  -'*l,m"M  ,1]  _  (l)  1 

By  a  modification  of  the  proof  of  Case  2  of  Theorem  (2.2.4), 


sup  r(u;IXI  .)  *  P  [Xr.i  >  ur.,+g?l 

uefl06i[i])  ,L  w[lj*  *  ’*  w(i]  ’  w(i+l]*‘ '  * ~  w[k] ss+0°  [l1  “  [l1  1 


v* 

l 


max(Y.,. . . ,Y.i  > 

1  r  ~  o//nJ 

where  Yj,...,Y^  are  i  independent  N(0,1)  r.v.'s.  To  make  the  minimal 
probability  of  coverage  y  (0  <  y  <  1)  we  set  g*  so  that 
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THEOREM :  The  upper  confidence  interval  of  (6.3.7)  on  y j 


(6.3.8) 


which  has  minimal  probability  of  coverage  y  has  maximal  prob¬ 


ability  of  coverage 


'  i  y 

l-[l-Yk‘1+1j  (i  -  0  <  Y  < 


The  lower  confidence  interval  of  (6.3.7)  on  y^  which 
has  minimal  probability  of  coverage  y  has  maximal  probability 
■  jlk-i+1 

of  coverage  1-  1-y*.  (i  *  l,...,k;  0  <  y  <  1) . 


The  proof  of  Theorem  (6.3.8)  is  similar  to  chat  of  Theorem  (6.3.7) 
and  will  be  omitted.  Note  that  (6.3.7)  and  (6.3.8)  also  hold  when  k  =  1, 
in  which  case  the  upper  and  lower  intervals  ony^  are  exact.  The  fol¬ 
lowing  table  illustrates  the  maximal  degree  of  overprotection. 
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Table  6.3.9. 


For  the  special  case  i  *  k,  Fraser  (1952),  p.  579,  pave  the  upper 
interval  on  y  ^  of  Theorem  (6.3.7)  as  one  with  probability  of  cover¬ 
age  at  least  y.  Fraser  proves  that  under  mild  conditions  an  upner 
confidence  interval  for  y  (k  ^  2) ,  with  probability  of  coverage 
y  (0  <  y  <  1)  for  all  y  e  does  not  exist. 

Our  results  above  extend  to  certain  location  parameter  families 
if,  instead  of  set-up  (1.3.1)  (normal  distributions),  we  take  set-up 
(2.1.1)  with  assumption  (2.1.2)  (a  location  parameter  family  with 
finite  mean) . 


THEOREM :  Suppose  we  have  location  parameter  populations 
as  in  (2.1.1)  and  assumption  (2.1.2)  holds.  For  any  i 
(1  <  i  <  k),  if  a^  *  1  (thus  a^  =  0  for  j  t  i)  then  the  risk 
(6.3.5)  ((6.3.6))  is  the  probability  that  our  upper  (lower) 
interval  does  not  cover  y  ^  and  is  maximized  over 


V  e  at 
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(6.3.10) 


i-1  terms  k-i+1  terms 


i  terms  k-i  terms 


v  *  v  -  • .  ,u  j)  (y  a  ^  [ i ]  *  *  *  ’ ,p  [i]  * *  • »+")) 

Thus,  for  any  y  (0  <  y  <  1)  an  upper  (lower)  confidence 
interval  of  minimal  probability  of  coverage  y  is  (-“.X^+ht) 


((X(i]“Ei’X“))  with  h l  =  -Gn‘1(l-yk'i+1|f)  ♦  Ef 


(g{  «  Gn'1(yi|f)  -  Ef).  If  gn(x|f)  is  symmetric  about  x  *  0 


this  becomes  h*  =  G  "1(yk'1+1|f)  *  E«  Cg?  «  G  "i(y1|f)  -  E  .) . 

l  n  1  1  f  n  v  1  f ' 

Proof :  Upper  Interval.  For  any  i  (1  ^  i  k),  if  a^  *  1,  a^  =0 

(j  t  i)  then  by  Theorem  (2.1.11) 

sup  r0i  ;i sup  P  (X  <u  rhJ] 


°  li1”  p  u  [Xr .  <y,.,-h*] 

[l]  [l]  1 


■ V»[i]-hl). 

M+  +■»  1  J 

where  »M(x)  =  pu  [X  [i]ix]  with  V  =  (-M, . . .  ,-M,y  ^  j , . . .  ,y  ^j)  .  Now 
FL,(x}+II  (x)  for  all  x  by  the  expression  for  (x)  given  in  the  proof 

X[i] 

of  Lemma  (2.2.5).  Thus 

sup  r(y:IM  ..)  =  P  [X,.,  <  yr.,-h*l 

yefi^  j.j)  h,U  M[l]  =  ",=M[i-l]S'a,,p[i]  =  *-'=M[k]  [l]  ~  [l1  1 


■  P[min(Y1,...,Yk_.+1)  <.  -h?+Ef] 


where  Y^,  •  •  •  are  (see  (2-1-7))  k-i+1  independent  r.v.'s  each 

with  d.f.  Gn(y|f).  It  follows  that  to  make  the  minimal  probability 


of  coverage  y  (0  <  y  <  1)  we  set  h?  so  that  h?  =  -C^-1 (l-y^"^+1 j f)  ♦  E^. 
Lower  Interval.  This  case  follows  in  a  similar  manner. 


THEOREM:  The  upper  confidence  interval  of  (6.3.10)  on 
which  has  minimal  probability  of  coverage  y  has  maximal  nrob- 


(6.3.11) 


ability  of  coverage 


1  l1 

k-i+1 


(i  =  1, . . .  ,k;  0  <  y  <  lj 


The  lower  confidence  interval  of  (6.3.10)  on  y  ^  which 
has  minimal  probability  of  coverage  y  has  maximal  probability 
nk-i+1 

of  coverage  1-  1-y1  (i  *  l,...,k;  0  <y  <  1). 


The  proof  of  Theorem  (6.3.11)  will  be  omitted.  Note  that  this 
result  implies  that  Table  6.3.9  provides  an  analysis  of  maximal  over¬ 
protection  for  our  location  parameter  case  as  well  as  for  the  normal 
case.  For  the  special  case  i  =  k,  Fraser  (1952),  p.  576,  gave  the  upper 
interval  on  y  of  Theorem  (6.3.10)  as  one  with  probability  of  coverage 
at  least  y.  Fraser  proves  that  under  mild  conditions  an  upper  confi¬ 
dence  interval  for  u  (k  >_  2),  with  probability  of  coverape  y 
(0  <  y  <  1)  for  all  y  e  does  not  exist  if  f(x-y)  satisfies  a 
condition  of  bounded  completeness.  V.Te  will  now  extend  this  result  to 
y[i]  (1  J  i  ±  k;  k  _>  2);  our  mild  conditions  are  slightly  stronger  than 
Fraser's. 

DEFINITION:  For  1  _<  i  <_  k,  let  g^  (x^ , . . .  jX^)  be  a  real¬ 
valued  function  such  that  for  any  j  (1  <  j  <  k) 


130 


(6.3.12) 


(6.3.13) 


(6.3.14) 


(6.3.15) 


N;>te  that 


(6.3.16) 


(6.3.17) 


gi(x1,...,xk)  <  g.(x1,...,x^_1,x;.+6,x^+1,...,xk) 
for  all  Xj,...,xk  e  R  and  6  >  0. 


DEFINITION;  For  1  <  i  <  k,  let 


*e,i*xl**  '  ”xk^  =  4 


1  if  ^(Xj 


[0  if  g.(xx 


.  ,xk)  >_  9 
. .X^)  <9. 


DEFINITION: 

Let  R0(yi-- 


For  any  i  (1  _<  i  _<  k)  for  £  =  1,2,...  let 

!the  ith  smallest  of  y .,..., y  if  £  >_  i 

)  }  "  X  X> 

=  '■ 

|+®  if  £  <  i. 

•  •  ,y£)  =  —  if  £  >_  l. 


DEFINITION:  For  1  _<  £  <_  k,  let 

S£  =  {(X], ....  ,xk) :  R.(x.,j7<£)  > 


x£  >  Ri-l(xj’^*)}* 


, (x. ,...,x.  )  is  a  monotone  non-decreasing  function  of 

0 ,1  X  K 

and  that  Sj,...,S^  are  disjoint  sets  whose  union  is  R  . 

ASSUMPTION:  Gn(y-0|f)  is  boundedly  comnlete  (each-sided), 
i.e.  Eg(X)  =  _£g(x)dGn(x-e| f)  =  0  for  a  dense  set  of 
0(<O  or>0)  and  |g(x)|  <M  imply  g(x)  =  0  (a.e.). 

THEOPFM:  Suppose  we  have  location  parameter  populations  as 
in  (2.1.1)  and  assumptions  (2.1.2)  and  (6.3.16)  hold.  Fix 
i  (1  *  i  1  k;  k  _>  2).  Then  an  upper  confidence  interval  for 
with  probability  of  covering  y  (0  <  y  <  1)  for  all 
y  e  and  satisfying  (6.3.12),  does  not  exist. 


m 


Proof:  Assume  that  Pj (r^ , . . .  .x^)  satisfies  (6.3.12)  and  yields  an 
upper  confidence  interval  for  y  ^-j  with  probability  of  covering 
Y  (0  <  y  <  1)  for  all  y  e  fiQ.  We  have 

y£  VM[i]  1  «  E<t>M^ti(V'-**V 

*  ^[ij.i^l . Vl^^r—V^n^Iil^fl^  ifyt  =  A[i 

*  E^Bjfj|j,i(yi»,,,*Vl*^l+l»,**,*k^  lfwt  =  u[i] 
where 

By(  [il»  i(Xl>  ,X^-1  ,X*+1’  *  *  ‘  'V  sJJ\i  fil .  i(xl'  *  * '  ,xk)dGn(x4‘w  [i]+Ef  * f) 


We  now  derive  conditions  on  the  function  . .  From  the  expression 

0,1 

above  it  is  seen  that  (if  y£  =  y[i]^ 

E[B^|(Xi,...)XH,Xw,...,Xk)  ••  y]  ■  0. 

Hence,  as  in  Fraser  (1952),  p.  580, 

®o,i^Xl’ ‘ * ’ ,X£-1,X£+1*’  *  * ,xk^  “  Y 

f  i) 

Using  the  above  condition  on  6^  :,  we  obtain  conditions  on  the  function 

0,1 


^o,i^xi *  *  *  *  »xj()  • 


Y  ~  3^  | (Xj , •  •  •  *X£+ j , •  •  •  »Xj.)  (a.e.l 


=  -Z*o,i^xl'“  *,xlPdGn^VEf^  ’ 

Consider  fixed  Xj , . . .  ,x£1  ,x^+1 , . . .  .x^.  Now  <f>Q  ^(Xj, . . . .x^)  is  a  mono¬ 
tone  function  of  x^,  and  since  it  is  a  characteristic  function  it  will 


have  the  following  form: 
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value  of  x£  at  which 


u(x1,...,x£l,x£+1,...,xk)  =  max 

fo  if  (Xj » *  x£  ^  u(Xj ,  •  •  •  »x£_ j *x£^j t  • • • > x^) 


*o,i(V*,"xk)  juars  *  Ri-l(xj*j^ 

from  0  to  1 


<^o,i(xl,',,’xk) 


1  if  u(Xj, . . . ,x0 ,x 


*-l*~t+l’****V  <x£  <w* 


Using  the  function  u^,...,^)  we  obtain 

R±_l  (xj , 


(6.3.18)  y 


However,  since 


»o,i(Xl’-"-VdCntVEflf)  * 


dGn(xt*Ef|f). 


(  •  •  •  ’  '  ‘  ’^]f^ 


Ri.lCXj.jVa) 

0  -  j  <>,o,i(xl,,,*,xk)dGn(VEflf) 


Ri-l(xj’^£) 

dGn(x£+Ef|f)  «  P[X£  IR.^Cx.JjOI)  +  Ef] 


then 

Gn  (1'Ylf)  '  Ef  1  u(Xi, . . .  ,  —  ,xR) 

1G^1(1-y+P[X£  <  R.^x.JM+E^)  -  Ef . 

The  inequality  on  uCXj,...,^)  implies  that  <|>o  ^(Xj , . . .  ,x^)  »  0  for 
(x1,...,xk)  e  S£  with  x£  <  G^Cl-yjf)  -  Ef.  This  is  true  for  Z  *  1, . . . ,  k; 

hence  $  .  (x,  ,...,x.)  »  0  if  R.(x.)  <  G"*(l-y|f)  -  Er.  Consider  now 

yo,iv  1’  *  k'  ij  n  v  1  '  f 
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(Xj,. . .  •  •  »X]^)  having  R^_j(Xj  ,  jjf  t)  <  (1-Y|f)j  in  (6.3.18), 

the  first  integral  vanishes  leaving 

<• 

Y«  J  dGn(x£+Ef|f). 

ufx1*-.-»xJl.j,x£+1,...,x,) 

Therefore  u(x, , . . . ,x£1>x£+1 , . . .  ,xk)  =  G~A  (  1-y |  f)  -  Ef,  if  »>i_1(x  ,  j^t) 

<  G"1 C1-y| c) •  From  this  equality  on  u(Xj , . . . >x£_j»x£+j, . . . ,xk) ,  we 
obtain  the  following  conditions  on  <J>  .  (x. , . . .  ,x. ) : 

0)11  Iv 


0  if  P..(x.)  <  G~ 1  ( 1-Y !  f)  -  E 


i'  J 


$ o  =  'S 


1  if 


n 

-1 


W  >  Gn  Cl"Y,f)  ‘  Ef 


Ri_l (xj D  <  G’1 Cl -Y I f) 
But  since  <J>q  i (Xj+6 , . . .  .x^+6)  is  monotone  in  6,  we  have 

0  if  R  (x.)  <  G'1(l-Y|f)  -  Ef 

<^0  i  ^X1 ,xk^  *  -1  I 

0,1  1  k  1  if  R.(x.)  >  G.  (1-y  f)  -  Er. 


(x.)  >  G  A (1-y | f) 
iK  y  n  v  1 


Therefore 


<  0  if  R.  (x.)  <  G-1  (l-y|  f)  -  E- 

„  ,v  V  1  1  J  n  f 

Bi (x. >  •  •  •  >x.  J  _ i 

11  K  >  G  if  R.(x.)  >  G  A(1-y  f)  -  E_. 

—  i  3  n  1  f 


Similarly 


“til  lf  VV  ‘  Giltl'Y|f)  *"m 'K» 

if  M**)  '  G^(l-vlf)  *  um  -  Ef 


?i (Xj i • • • >xk) 


-M[i]  x.  v,n  V* -  ^fi]  **f 

This  completely  determines  gi(xJ, . . . ,xk) :  gi(xJ , . . . ,xk)  =  ^(x^) 


-  G'^I-yIH  +  E...  But  we  know  that  a  constant  added  to  this  yields 
n  1  f 


APPENDIX  A.  MAXIMA  AND  MINIMA  OF  REAL-VALUED  FUNCTIONS 
OF  n  REAL  VARIABLES 
A-l.  n  =  2 


Although  the  case  n  =  2  is  included  in  the  case  n  ^  2  of  Section 
A-2,  it  will  be  convenient  to  have  stated  separately  the  results  and 
notations  of  this  special  case.  (Note  that  some  authors,  e.g. 

Kaplan  (1952),  p.  126,  state  these  results  in  a  somewhat  more  cumbersome 
manner.) 

THEOREM :  Let  f  have  continuous  second-order  partial 
derivatives  on  an  open  set  S  in  R2.  Let  (x°,x°)  e  S  be  such 


that 


3f(x1,x2) 


3x, 


9f(x1,x2) 


,  0  0. 


3  x. 


=  0, 


,  o  o. 

(Xj,x2) 


and  let 


92f(x1,x2) 


3x2 


(xj,x®) 


(A. 1.1) 


B  = 


3  2f(xJ,x2) 


3  Xj3  x2 


,  o  os 
,x2) 


C  = 


3  2f(x1,x2) 


3x2 


(xj,x°)  . 
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Then  (x°,x°)  is 

(i)  a  relative  minimum  if  B2-AC  <  0,  A  >  0 

(ii)  a  relative  maximum  if  B2-AC  <  0,  A  <  0 

(iii)  of  undecided  nature  if  B2-AC  ■  0;  and 

(iv)  a  saddle  point  if  B2-AC  >  0. 


APPENDIX  A.  MAX  IMA  AND  MINIMA  OP  REAL-VALUED  FUNCTIONS 
OF  n  REAL  VARIABLES 
A-2.  n  >  2 


Even  in  Hancock  (1960)  and  Apostol  (1957)  the  presentation  of  the 
theory  of  maxima  and  minima  is  not  as  complete  as  we  need  (e.p.,  in 
order  to  show  in  total  the  asymptotic  nature  of  (X,...,X)  in  Section 
5.1).  We  therefore  present  a  summary  gathered  from  several  sources. 
THEOREM :  Let  f  have  continuous  second-order  partial 
derivatives  on  an  open  set  S  in  pn.  Let  (x°,...,x°)  e  S  be 
such  that 


3f(Xj,...,  xn ) 


dx. 


(i  ■  1 , . . .  ,n) , 


,  o  o. 

(Xji • • • 


and  let  Q  ■  (d. .)  where 
x  ij 


d. 


IJ 


(A. 2.1) 


(i,j  *  1, .. . ,n) . 


,  0  o. 

(Xj....,xn) 


Then  the  real  symmetric  matrix  0  is  either 

(i)  positive  definite,  in  which  case  (x°,...,x°)  is 
a  relative  minimum; 

(ii)  negative  definite,  in  which  case  (x°,...,x°)  is 
a  relative  maximum; 

(iii)  semi-definite,  in  which  case  the  nature  of 
(x°,...,x°)  is  undecided;  or 

(iv)  indefinite,  in  which  case  (x°,...,x°)  is  a  saddle 
point. 
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Proof:  In  addition  to  nrcviously-ci ted  references,  sec  Courant  (1966), 

pp.  204-208. 


(A. 2. 2) 


T! IFO’M-M:  A  real  symmetric  matrix  0,  having  eigenvalues 
^  j ,  .  .  .  ,X  n  (say)  is 


(i)  positive  definite 

iff  X.  > 

0  (i 

*  1,  •  • 

.  ,n) ; 

(ii)  negative  definite 

iff  x.  < 

0  (i 

•  1... 

.  ,n) ; 

(iii)(a)  positive  semi-definite 

iff  X  .  > 

l  - 

9  (i 

*  1 , . . 

.  ,n) 

and  at  least  one  X  .  =  0- 

J 

(b)  negative  semi -definite  i f f  X  <  0  (i  «  1 , . .  .  ,n) 

and  at  least  one  X  .  =  0-  and 

3 

(iv)  indefinite  iff  at  least  one  X.  is  positive  and  at 
least  one  X  .  is  negative. 


Proof :  Recall  that  the  eigenvalues  of  a  matrix  0  arc  t’'e  n  roots  of  the 
equation  | O-X I |  =  0,  and  see  ''edderburn  (1964),  n.  02. 


THr.OrTM;  For  the  real  symmetric  matrix  0,  let  A  *  det(O) 

and  A  =1.  Let  A  .  be  the  determinant  of  0  with  its  last 
o  n-t 

t  rows  and  columns  deleted.  (Vote  that  A  *  A.)  Then  0  is 

n 


(A. 2. 3) 


(i)  positive  definite  iff  Aq,Aj,...,A  are  positive’ 

(ii)  nceative  definite  iff  Ao>Aj,...,An  are  alternately 
positive  and  m;,ativc; 

(iii) (a)  positive  semi -definite  iff  all  principal  minors 


of  0  arc  >0  and  A  =  0; 
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(b)  negative  semi -definite  iff  all  prir.cinnl  minors 
of  0  arc  >0(<0)  if  their  order  is  even  (odd),  and 
A  ■  0;  or 

(iv)  indefinite,  otherwise. 

Proof :  For  (i)  and  (ii),  see  (e.p.)  Narayan  (1962),  np.  165,  167.  (Motc 
that  the  reference  cited  by  Anostol  (ln57)  is  inadequate;  it  proves  a 
weaker  theorem  which  utilizes  more  than  the  leading  nrincinal  minors  of 
Q.) 

For  (iii)(a),  from  Browne  (1958),  we  know  0  is  positive  semi- 
definite  iff  all  principal  minors  of  0  are  >p  (see  pn.  1 ?o_ 121,  Theorem 
46.5).  If  0  is  to  he  positive  semi-definite  but  not  definite,  then 
the  condition  should  also  specify  A  =  0.  (This  modification  holds  for 
the  “>  implication  by  the  well-known  result  A  =  Xj...^,  Faddceva 

(1959),  p.  14.  The  <«•  implication  is  clear.)  V/e  use,  of  course, 

Theorem  (A. 2. 2)  . 

For  (iii)(b),  note  that  for  any  matrix  A  of  order  i,  det(-A) 

«  (-l)1det(A),  and  that  0  is  negative  semi-definite  iff  -0  is  positive 
semi-definite. 


Note .  A  condition  such  as 
suffice  for  (iii)(a)  of  Theorem 

0 


(A.2.3) . 

0  o' 

3 

0  -1J 


, A  >  0  and  A  *  0"  will  not 

n  -  - 

^or  examnle,  consider 


A  P 
B  C 


=  AC-32,  Aj  =  A,  A^  = 


Note.  If  n  =  2,  0 


A 


A  and  o  is 


140 


(i)  positive  definite  iff  B2-AC  <  0,  A  >  0; 

(ii)  negative  definite  iff  B2-AC  <  0,  A  <  0; 

(iii) (a)  positive  semi-definite  iff  B2-AC  -0,  A  >_  0,  C  ^  0; 

(b)  negative  semi-definite  iff  B2-AC  ■  0,  A  0,  f  0;  and 

(iv)  indefinite  iff  {B2-AC  ■  0,  A  >  0,  C  <  0}  or 
(B2-AC  -  0,  A  <  0,  C  >  0}  or  (B2-AC  >  0). 

Here,  we  have  reduced  the  number  of  undecided  cases  ((iii)  cases) 
"beyond"  those,  namely  B2-AC  ■  0,  named  in  virtually  all  texts.  (The 
cases  separated  out  belong  to  (iv)  and  are  therefore  saddle  points.) 
However,  by  a  consideration  of  signs  it  is  easy  to  see  that  the  sets 
(B2-AC  *  0,  A>  0,  C  <  0}  and  (B2-AC  *  0,  A  <  0,  C  >  0}  are  empty.  (The 
reason  for  this  is  the  need  to  have  at  least  one  positive  and  one 
negative  eigenvalue,  thus  exhausting  the  supply  of  eigenvalues  when 
n  -  2.) 


apphnoix  b.  oiSTf>i  but  ions  op  various  functions  of 


The 


where  f— 


(see  (5. 
ordered 


(B.1.1) 


CURTAIN  r*A"POH  VAPIABIJS 


3-1.  JOINT  D!ST"inUTION  Or  X  , . . .  ,X ^ 


joint  density  of  is 

j  (yj . •  •  •  »yk)"fx  (yk)  (yj  e  R;  *  ■  i . k) 

’  ,Ak  1  k 

(•)  is  the  M(u.,o2/n)  density  function  (i  ■  l,...,k) 

L  1 

1.1)) .  It  is  well-known  that  then  the  joint  density  of  the 
(.  (i  =  1,. i.e.  of  X .  .<7^  ,  is 

ftr  y"  (x.,...  ,X.  ) 

*|1|  •*  *  * ’  fk] 

fX1,...,Xk(x0(l)**",XB(k))’  Xl-  *-Xk 


0 

l  (•/n/o)kt 

ficSk 

0 


,  otherwise 


xBCl)nJ  1 


\  c//  n 


xe(k)^k 


*  V*  •  *-xk 


o//  n 

,  otherwise 


:^x 

1  • 

N— / 

u 

_ J 

A 

XB(h)"lJ  (h  1 

\  o//  n 

•  •  •  9 

l  o//n 

,  X  J  <_.  .  .  <X|. 


,  otherwise. 
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APPENDIX  B.  DISTRIBUTIONS  OF  VARIOUS  FUNCTIONS  OF 


CERTAIN  RANDOM  VARIABLES 


B-2 .  LIMIT  DISTRIBUTION  OF  X[ : j . • • • 


The  limiting  distribution  of  X|j j » •  •  •  j  (under  certain 
parameter  configurations)  is  of  interest  to  us.  Let  (A^,  n  >_  1} 
and  (Bn,  n  >  1 }  be  sequences  of  events  on  some  probability  space 

k 

(which  may  depend  on  n)  .  Lot  a  ■  (a ^ , . . . , a^ )  c  R  be  fixed,  and  denote 
the  vector  (uj+ajO/Zii, . . .  .u^+a^o/Zn)  by  y  ♦  ao//n. 


(B.2.1) 


LEMMA:  If  j^m  P^B^)  *  1,  then  (if  either  of  the  following 
limits  exists)  VA^)  -  Pn(An). 


Proof :  Suppose  l^m  Rn(Bn)  *  1-  Then  by  taking  limits  in  pn(Rn) 

<  P  (AUB  )  <  1  we  find  lim  P  (AUB  )  «  1,  and  hence 
—  nnn—  n-*00  n  n  n 

lim  (P  (B  )  -  P  (A^B  )}  »  0.  Taking  limits  in  P  (A  1  )  «  P  (A  ) 
n+“  nv  n  nv  n  n7  n  n  n  nv  n' 

+  (pn(Bn)  •  Pj^fA^B^}  yields  our  result. 


DEFINITION:  For  y  c  Uq,  let  p(n|y)  =  PufX(i)  <  <  X(k)^» 

(B.2.2)  where  X^j,...,X^)  are  as  *n  definitions  (1.3.13)  and 
(1.3.14) . 
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LEMMA  :  Let  0  ■  (p:  u  c  fiQ,  p  j-w  .  ,uv°u  ^j} .  For  all 

(B.2.3)  w  c 

W*  P'"l“*a°//")  ■  WE  r„.aa/^|ir(l)  '  •••  *  *(k))  ‘  >• 

Proof:  1.  Suppose  that  u  e  Q(^)ft0.  Then  for  all  n  large  enough, 
u  ♦ao//ti  e  0(j0o0.  Then  the  are  independent  and  is 

the  sample  mean  of  n  i.i.d.  N(p^|  +  a^a//n,o2 )  r.v.'s.  The 

characteristic  function  of  a  N(m,o2)  r.v.  is  (see,  e.g.,  Parren  (I960), 
p.  221)  f(t)  ■  exptitm-  jt2o2}.  Thus, 


K-  (t).BeltX«). 

(j) 


.  a  1  t2  2 

ltum  lta;^-  2^ 

e  e  e 


,  i  r  i  I 

so  that  lim  r7  (t)  «  e  1  .  It  is  then  well-known  (see,  e.g.,  Wilks 

X(j) 

(1962),  p.  124,  5.4.1a)  that  X^  converges  in  probability  top^j 
(j  ■  l,...,k).  Thus,  since  the  X^  are  independent,  it  is  clear  that 
the  probability  that  (X^  converges  top^j  (j  =  l,...,k)}  approaches 
1  as  n*«.  However,  by  Lemma  (B.2.1) 

p+ao//nl  (1)  (k)J 


(B.2.4) 


ni«  +ao//n  (1) 


<  . . .  <  X 


00’ 


lx(l)-P[l]l  c  e 


*  I X (k) -VJ  [k]  I  * 
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for  any  c  >  0.  If  we  choose  2c  <_  nin  (p r  ,1-pf ,  |) ,  then  the  r  h.s. 

1  <i  <  j  j-  k  m  llJ 

of  (B.2.4)  equals 

PU4ao//nf^(l)‘U[l]l  '  E  *  ‘‘  *^(k)'pfk]l  < 

which  is  1  since  P[X^  converges  to  p  (j  ■  l,...,k)]  approaches  1  as 

n*°°. 

2.  Suppose  that  u  c[D(?<)]crt0.  (F.ventual ly  u ♦ao//n  e  O^Q Cj^)  »  or 
The0  there  are  £  distinct  values  in  (u  ^♦ajO/Zn, . . . , 

u  [j^ j ♦a^o// n }  (1  ^  £  <_  k-1)  and  (see  (1.3.14)) 

Pu+ao//nf*(l)  <  •  •  •  *  ^(k)1 


"  Pu*ao//n^X(i1)  *  X(i]L+l)'  X(i2)  <  X(i2*l) 


'  •••  *X(i £_j)  *  X(i£_i*l)1* 


However,  the  result  will  not  follow  as  before  since  min  ,.,-e  ri) 

l£i  <j<_k  UJ  11 J 

=  0  here.  It  can  be  seen  (e.g.,  consider  the  case  k  «  2)  that  the  limit 
^  1  as  m-°°.  (In  fact,  it  depends  on  a.) 


LF.MMA :  For  u  c  0nn(j<),  as  m-°° 


(u+aa//n ) 

Fy"  7  (x. ,  .  .  .  ,  X.  ) 

(B.2.5)  X[l]  ’  ’  '  ’  ,A(k]  1 


i  *1  <*  *  1 . k»* 


Proof : 


(u+ao//n) 

is  PV  7  (xi  *  *  •  •  »xv  J 

A[il  *  —  *  [kl 
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lim{p(n|y  ♦an//n) 


n* 


,Pp^ao//nfX[l]  -  V  *'•  *X[k]  -  xk 


X(l)  " 


<  V)> 


♦  (l-p(n|g+ao//n))* 

*  Py  ♦ao//n^X  [  1]  -  V  "•  ,X[k]  -  Xk  I  not  (X(1)  <  •'*  <  X(k)^l 

"ni£  Pu*ao//n'fX[l]  -  V  ,X[k]  -xk:  X(l)  <  *•*  *  X;k)^ 

n~*"»  p«-ao/vnl  (1)  -  1*  (k)  —  k’  (1)  (k)J 

■  limP  ,/-[XM,  <  x,,  ...  .X,.  .  <  x.  ] . 

n-f®  g+ao//nl  (1)  -  1  *  (k)  -  kJ 

Here  the  second  equality  follows  from  Lemma  (B .2.3),  while  the  last 

equality  follows  from  Lemmas  (B.2.3)  and  (B.2.1). 


(B.2.6) 


LFMMA :  As  n-*-«,  ifp*aa//n  e  g Jl(t)  then 

P  */“ [X,,*  "  X1  *  •••  »X,...  *  X.  ] 

g+ao//nl  (1)  -  1*  '  (k)  -  V. 


Pu  rx(l)  -  V 


Proof :  As  n*®, 

Pli.ac//n(*(l)  -  X1 . *(k)  -  V 

'  Pg.ao//Sl*(l)-V//"-xrV//"-  •••  -?(k)-ak0//"  iW^ 

■  p„  l?(i)  i  xrai0//" . 5r(k)  i  VV//"1 

Pj,  I*(l)  i  X1 . *(k)  -  xk^  * 

The  second  equality  follows  because,  whenu+ao//n  e  Gnft(^),  X^  is 
N(u  ^j+a^o//n,  o2/n)  iff  X^^-a^o//n  is  N(U|-^,o2/n)  (i=l,...,k). 
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(B.2.7) 


DEFINITION:  Let  *(Zj,...,z  )  denote  the  d.f.  of  the 
order  statistics  in  a  sample  of  size  5  from  a  N(0,1)  population. 


(B.2.8J 


THEOREM :  As  ih«,  if  g  c  ©n^(^)  then 


(y  +ao//  n) 
F/n^ 


^  n  —  _  |/n  _____  va« 

— Cx [1]  -u  [i] ' ajo//n) , . . . (X |-kj  -y  ^  -a^/Zn) 


(Xj * • • • »x^) 


X 

-*■  n  *(x.). 
i=l 

Proof :  This  follows  from  Lemmas  (B.2.S)  and  (B.2.6). 


COROLLARY :  As  n+®,  if  y  e  then 


(B.2.9) 


(X1,...,xk)  -  M  4(X.). 

ol  [irw[l]J,,,,’“5 1  [k]'u[k]J 


k_ 

II 

i=l 


(x, » > « • »xk) 


THEOREM:  If  y  e  0n[o(^)]  then 
(y+ao//n) 

(B.2.10)  lim  V  r-  _  j—  _  v^. 

n^~  — CXj^-Ufjj^jo/Zn),...,— (X^^-yjjcj^o/^ 

depends  on  a. 

Proof:  (A  hint  of  this  dependence  was  given  in  part  2  of  the  proof  of 
Lemma  (B.2.3).)  Suppose  k  =  2,  a  -  (a^,a2)  with  <_  a^,  and  let 

Yj,Y2  denote  i.i.d.  N(0,1)  r.v.'s.  Then  Ujjj  =  p^2]  an<* 

(u+aa/Zn) 


/n  /n  _ 

~§C3r[l]  _y  [1]  "aiCT//n) »  ~aC^[2]_u  [2]  'a2o//") 


(Xj,x2) 


■  P„.a^[->in(i‘l-’r2>-“[l]-alo/’,;i)  ixl’ 

>^(nax(X1,X2)-iJj]j-a2o//n)  <  xj 
*  P[min(Y1,Y2+(a2-a1))  ^Xj,  maxCYj- (a^a^  ,Y2)  _<  x?] . 

For  a2  -  aj  =  0,  this  is  0(x1>x2).  However,  for  a2  »  it  is  approx¬ 
imately  <t(Xj)  4>(x2) ,  and  therefore  depends  on  a. 


APPENDIX  B.  DISTRIBUTIONS  OF  VARIOUS  FUNCTIONS  OF 


CERTAIN  RANDOM  VARIABLES 

B-3.  JOINT  DISTRIBUTION  OF  *  *  *  * ** fk] [k-1] 


From  the  joint  density  of  given  at  (B.1.1),  we  find 


that  (for  Xj  <_  X£) 


f y  y"  (X,  ,xj 

[1]’  (2]  1  1 


2  IT  a2 
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e  >-v 


j/Zn 


*2^  [2] 


[  o/Zn 


+  e 


_2_  !ii 
o/Zn 


12 


x,  -v 


1 ZI21 


*/Zn  J 


so  that  (for  y  >  0),  setting  n  =  u  ^2j  ”  u  [l]  ’ 


%  (y)  = 

X[2]  "[l] 


Ifx  x  (x,y+x)dx 

X[1]’X[2] 


2tto2 


2 

ie 


x"w0) 

2 

y-x"yr2i 

2' 

o/ZnT 

▼ 

s/Zn  . 

(B.3.1) 
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Since,  via  completing  the  squaro, 


„  -  j[(x^a)2>(x>b)2] 

Je  dx 


1_ 

2 


x^ 


l 


(«♦*>) 
_ 2_ 

1//2 


2 


e 


(a-b)2/4 

dx 


j(a-b) 


it  follows  from  (B.3.1)  that 
THEOREM:  With  n  15  M 


[2)  •  “nr  for  y  i° 


(B.3.2) 


fv-  .v  (y) 
X{2]  X[l] 


2tV  IT 


1 

T 


1° 


//JT 


♦  e 


y*o 


■>2 


(o//  n 


REFERENCES 


Abramowitz,  M.  and  Stcgun,  I.  A.  (Editors):  Handbook  of  Mathematical 
Functions  With  Formulas ,  Graphs,  and  Mathematical  Tables,  U.  S. 
Government  FrTntinp.  Office,  Washington,  D.  C.,  June  1964. 

Alam,  K.:  "A  two-sample  estimate  of  the  largest  mean,"  Annals  of  the 
Institute  of  Statistical  Mathematics  (Tokyo),  Vol .  19,  No.  2 
(1967),  pp.  271-283. 

Apostol,  T.  M.  :  Mathematical  Analysis,  Addison-Wesley  Publishing  Co., 
Inc.,  Reading,  Massachusetts ,  1957  (Third  Printing,  December  1960). 

Bahadur,  R.  R.  :  "Sufficiency  and  statistical  decision  functions," 
Annals  of  Mathematical  Statistics,  Vol.  25  (1954),  pp.  423-462. 

Bahadur,  R.  R.  and  Goodman,  L.  A.:  "Impartial  decision  rules  and  suf¬ 
ficient  statistics,"  Annals  of  Mathematical  Statistics,  Vol.  23 
(1952),  pp.  553-562. 

Bechhofer,  R.  £.:  "A  single-sample  multiple  decision  procedure  for 

ranking  means  of  normal  populations  with  known  variances,"  Annals 
of  Mathematical  Statistics,  Vol.  25  (1954),  pp.  16-39. 

Bechhofer,  R .  E.:  "Design  of  experiments,"  a  course  offered  at  Cornell 
University,  Ithaca,  New  York,  Fall  Term,  1964. 

Bechhofer,  R.  E.,  Kiefer,  J.  and  Sobel,  M. :  Sequential  Identification 
and  Ranking  Procedures  (with  special  reference  to  Koopman-Darmois 
populations) ,  University  of  Chicago  Press,  Chicago,  Illinois,  1968. 

Bechhofer,  R.  E.  and  Sobel,  M. :  "Non-parametric  multiple-decision 

procedures  for  selecting  that  one  of  k  populations  which  has  the 
highest  probability  of  yielding  the  largest  observation  (prelimi¬ 
nary  report),"  Abstract,  Annals  of  Mathematical  Statistics, 

Vol.  29  (1958) ,  p.  325.  . 

Berk,  R.  H. :  "Zchna,  Peter  W. .  Invariance  of  maximum  likelihood 

estimators,"  Review  #1922,  Mathematical  Reviews,  Vol.  33  (1967), 
pp.  342-343. 

Blumenthal,  S.  and  Cohen,  A.:  "Estimation  of  two  ordered  translation 
parameters,"  Annals  of  Mathematical  Statistics,  Vol.  39  (1968a), 
pp.  517-530. 

Blumenthal,  S.  and  Cohen,  A.:  "Estimation  of  the  larger  translation 
parameter,"  Annals  of  Mathematical  Statistics,  Vol.  39  (1968b), 
pp.  502-516. 


151 


Blumenthal ,  S.  and  Cohen,  A,:  "Estimation  of  the  larger  of  two  normal 
means,"  Journal  of  the  Americun  Statistical  Association,  Vol.  63 
(1968),  pp. 

Browne,  0.  T. :  Introduction  to  the  Theory  of  Determinants  and  Matrices, 
University  of  North  Carolina  Press,  Chapel  Hill,  vorth  Carolina, 
1958. 

Chambers,  M.  L.  and  Mack,  C.:  "Confidence  limits  for  the  minimum  of 
two  normal  means;  a  new  inference  principle,"  New  Journal  of 
Statistics  and  Operational  Research,  Vol.  2  (1966) ,  pp.  14-27. 

Courant,  R.:  Differential  and  Integral  Calculus,  Vol .  II  (translated 
by  E.  J.  McShane),  Interscience  Publishers,  Inc.,  New  York,  1°36 
(Reprinted  1966) . 

Dudewicz,  E.  J. ;  The  Efficiency  of  a  Nonparametric  Selection  Procedure : 
Largest  Location  Parameter  Case ,  unpublished  M.S.  thesis,  Cornell 
University,  Ithaca,  New  York ,  February  1966.  (Reprinted  as 
Technical  Report  No.  14,  Department  of  Operations  Research, 

Cornell  University,  Ithaca,  New  York,  December  1966.) 

Dudewicz,  E.  J.:  "The  Robustness  of  a  Selection  Procedure  of  Bechhofcr," 
Technical  Report  in  preparation,  Department  of  Operations  Research, 
Cornell  University,  Ithaca,  New  York,  1968. 

Eaton,  M.  L.:  "Some  optimum  properties  of  ranking  procedures,"  Annals 
of  Mathematical  Statistics,  Vol.  38  (1967),  pp.  124-137. 

Faddeova,  V.  N.  :  Computational  Methods  of  Linear  Algebra,  Dover  Pub¬ 
lications,  Inc.,  New  York,  1959. 

Feller,  W. :  An  Introduction  to  Probability  Theory  and  Its  Applications, 
Vol .  1^  (Second  Edition),  John  Wiley  f«  Sons,  Inc.,  New  York,  1957. 

Ferguson,  T.  S.:  Mathematical  Statistics:  A  Decision  Theoretic  Ap¬ 
proach  ,  Academic  Press  Inc.,  New  York,  1967. 

Fisz,  M. :  Probability  Theory  and  Mathematical  Statistics  (Third  Edi¬ 
tion)  ,~JoTm  Wiley  5  Sons,  Inc . ,  New  York ,  1963. 

Fraser,  D.  A.  S.:  'Confidence  bounds  for  a  set  of  means,"  Annals  of 
Mathematical  Statistics,  Vol.  23  (1952),  pp.  575-585. 

Graybill,  F.  A.:  An  Introduction  to  Linear  Statistical  Models,  Volume  I, 
McGraw-Hill  Book  Company,  Inc.,  New  York,  1961. 

Gupta,  S.  S.:  "On  a  decision  rule  for  a  problem  in  ranking  means," 

Mimeograph  Series  No.  150,  Institute  of  Statistics,  University  of 
North  Carolina,  Chapel  Hill,  North  Carolina,  May  1956. 


it 


A 


152 


Gupta,  S.  S.:  "On  some  multiple  decision  (selection  and  ranking) 
rules,"  Tcchnomctrics ,  Vol .  7  (1965),  pp.  225-245. 

Hall,  W.  .1.:  "Most  economical  multiple-decision  rules,"  Annals  of 
Mathematical  Statistics,  Vol.  2f'  (1958),  pp.  1079-1094. 

Hall,  W.  J.:  "Tlie  most-economical  character  of  some  Rechhofer  and 

Sobel  decision  rules,"  Annals  of  Mathematical  Statistics,  Vol.  30 
(1959),  pp.  964-969. 

Hancock,  H.  :  Theory  of  Maxima  and  Minima,  Dover  Publications,  Inc., 

New  York,  1960. 

Harter,  H.  L.:  'Expected  values  of  normal  order  statistics," 
Biometrika,  Vol.  48  (1961),  pp.  151-165. 

Hodgman,  C.  D.  (Editor):  C.  R^.  C.  Standard  Mathematical  Tables 

(Twelfth  Edition),  Chemical  Rubber  Publishing  Company,  Cleveland, 
Ohio,  1959 

Hogg,  R.  V.  and  Craig,  A.  T. :  Introduction  to  Mathematical  Statistics 
(Second  Edition),  The  Macmillan  Co.,  New  York,  1965  (Second 
Printing  1965) . 

Kaplan,  W.  :  Advanced  Calculus,  Addison-Weslcy  Publishing  Co.,  Inc., 
Reading,  Massachusetts,  1952  (Fifth  Printing,  July  195°). 

Katz,  M.  W.:  "Admissible  and  mininax  estimates  of  parameters  in  trun¬ 
cated  spaces,"  Annals  of  Mathematical  Statistics,  Vol.  32  (1961), 
pp.  136-142.  ‘  " 

Katz,  M.  W. :  "Estimating  ordered  parameters,"  Annals  of  Mathematical 
Statistics ,  Vol.  34  (1963),  pp.  967-972. 

Kendall,  M.  G.  and  Stuart,  A.:  The  Advanced  Theory  of  Statistics, 

Vol .  1_:  Distribution  Theory  (Second  Edition),  Hafner  Publishing 
Co.,  New  York,  1963. 

Kiefer,  J.:  'Invariance,  minimax  sequential  estimation,  and  continuous 
time  processes,"  Annals  of  Mathematical  Statistics,  Vol.  28 
(1957),  pp.  573-601. 

Lai  Saxena,  K.  M.  and  Tong,  Y.  L. :  "Interval  estimation  of  the  largest 
mean  of  k  normal  populations,"  Abstract,  Annals  of  Mathematical 
Statistics,  Vol.  39  (1968),  pp.  704-705. 

Lawton,  W.  II.:  "Concentration  of  random  quotients,"  Annals  of 
Mathematical  Statistics,  Vol.  39  (1968),  pp.  466-480. 

Lehmann,  E.  L. :  "Ordered  families  of  distributions,"  Annals  of 
Mathematical  Statistics,  Vol.  26  (1955),  pp.  399-419. 


153 


Lehmann,  E.  L. :  "On  a  theorem  of  Bahadur  and  Goodman,"  Annals  of 
Mathematical  Statistics,  Vol.  37  (1966),  pp.  1-6. 

Lofeve,  M. :  Probability  Theory  (Third  Edition),  0.  Van  Nostrand  Co., 

Inc.,  Princeton,  New  Jersey,  1963. 

Mahamunulu,  D.  M.:  "Some  fixed-sample  ranking  and  selection  problems," 
Annals  of  Mathematical  Statistics,  Vol.  38  (1967),  pp.  1070-1091. 

Narayan,  S.:  A  Text  Book  of  Matrices  (Fourth  Edition),  S.  Chand  ft  Co., 
Delhi,  India,  1962. 

Parzen,  E. :  Modern  Probability  Theory  and!  It£  Applications ,  John  '"i  1  ey  f 
Sons,  Inc.,  Mew  York,  1960  (Fourth  Printing,  March  1963) . 

Paulson,  F..:  "A  sequential  procedure  for  selecting  the  population  with 
the  largest  mean  from  k  normal  populations,"  Annals  of 
Mathematical  Statistics,  Vol.  35  (1964),  pp.  174-180. 

Reitsma,  A.:  *  On  approximations  to  sampling  distributions  of  the  mean 

for  samples  from  non-normal  populations,'  Annals  of  Mathematical 
Statistics ,  Vol.  34  (1963),  pp.  1308-1314. 

Robertson,  T.  and  Waltman,  P.:  "On  estimating  monotone  parameters," 
Annals  of  Mathematical  Statistics,  Vol.  39  (1968),  pp.  1030-1039. 

Teichroew,  D. :  Probabilities  Associated  with  Order  Statistics  in 
Samples  from  Two  Normal  Populations  with  Equal  Variance, 

ENASR  no.  ES-3,  Cherical~Corns  Engineering  Agency,  Engineering 
Statistics  Unit,  Army  Chemical  Center,  Maryland,  December  7,  ln55. 

Teichroew,  D.:  "Tables  of  expected  values  of  order  statistics  and 
products  of  order  statistics  for  samples  of  size  twenty  and 
less  from  the  normal  distribution,"  Annals  of  Mathematical 
Statistics ,  Vol.  27  (1956),  pp.  410-426. 

Tippett,  L.  H.  C.:  'On  the  extreme  individuals  and  the  range  of 
samples  taken  from  a  normal  population,"  Biometrika,  Vol.  17 
(1925),  pp.  364-387. 

Wadsworth ,  G .  P .  and  Bryan ,  J .  G . :  Introduction  to  Probability  and 
Random  Variables,  McGraw-Hill  Book  Company,  Inc.,  New  York,  1960. 

Wedderburn,  J.  H.  M.:  Lectures  on  Matrices,  Dover  Publications,  Inc., 

New  York,  1964. 

Weiss,  L. :  '  On  estimating  scale  and  location  parameters,"  Journal 

of  the  American  Statistical  Association,  Vol.  58  (1963), 
pp.  658-659. 


154 


Weiss,  I..  and  Wolfowitz,  J.:  'Generalized  maximum  likelihood 

estimators,"  Teorija  Verojatnostei  i  ee  Primenenija,  Vol.  11 
(1966),  pp.  68-93. 

Weiss,  L.  and  Wolfowitz,  J.:  "Ccncr.il i zed  maximum  likelihood 

estimators  in  a  particular  case,"  Teorija  Verojatnostei  i_  ee 
Primenenija ,  Vol.  12  (1967a),  to  appear. 

Weiss,  L.  and  Wolfowitz,  J.:  "Maximum  probability  estimators," 

Annals  of  the  Institute  of  Statistical  "athematies  (Tokyo), 

Vol.  19,  No.  2  (1967b),  pp.  193-206. 

Weiss,  I,,  an  1  Wolfowitz,  J.:  '  estimation  of  a  density  function  at 

a  point,"  Zeitschrift  fur  V.'ahrscheinlichkeitstheorie  und 
Verwandte  Cebiete,  Vol.  7  (1967c),  np.  327-335. 

Weiss,  L.  and  Wolfowitz,  J.:  "Maximum  probability  estimators  with  a 

general  loss  function,"  to  appear  in  the  Proceedings  of  the  Inter 
national  Symposium  on  Probability  and  Information  Theory,  held 
April  4-5  (1968)  at  McMaster  t'niversity,  Hamilton,  Ontario,  Canada 

Will  s,  S.  S.:  Mathematical  Statistics,  John  Wiley  f.  Sons,  Inc., 

New  York,  1962  (Second  Printing  with  Corrections,  1963). 

Zehna,  P.  W. :  "Invariance  of  maximum  likelihood  estimators," 

Annals  of  Mathematical  Statistics ,  Vol.  37  (1^66),  p.  744, 


Unclassified 


Security  Claasiflcation 


DOCUMENT  CONTROL  DATA  •  RAD 


(Security  c  laaaillcation  of  tltla  body  ot  abatract  and  tndanlng  annotation  mull  ba  an  I  a  rad  *4ian  tha  ovatall  raport  $a  c  laa  attiad) 


1  ORI G IN ATING  ACTIVITY  (Corporata  author) 


Department  of  Operations  Research 
College  of  Engineering,  Cornell  University 
Ithaca,  New  York  14850 


!•  REPORT  SECURITY  c  L  AMI  *  ic  A  T  ION 

Unclassified 


1  nePOKT  TITLE 


ESTIMATION  OF  ORDERED  PARAMETERS 


4  DESCRIPTIVE  NOTES  (Typa  ot  raport  and  tnclualva  dataa) 

Technical  Report,  February  1969 


5  AU  THORfS;  (Laat  nmma.  tint  nama.  Initial) 


Edward  J.  Dudewicz 


•  REPO  AT  DATE 

February  1969 


Eft  CONTRACT  OR  GRANT  NO 

Nonr  401(53) 
otfftHXKKKKK 

N00014-68A-0091 

e  CP-7798 


7*  TOTAL  NO  OP  P  A  Q I  ■  76  NO  OP  AtP6 

154  62 


f*  OPItlNATOP'l  AtPOAT  NUM61*Jj 

Technical  Report  No.  60 


ID  OTHER  REPORT  NOffJ  (A  ny  othar  numbara  that  may  ba  aaat0>ad 
m la  raport) 


<*  GP-8958 


10  A  VA  IL  ABILITY/LIMITATION  NOTICEI 


Distribution  of  this  document  is  unlimited 


12  IPONIOAINO  MILITARY  ACTIVITY 

Center  for  Naval  Analysis 
University  of  Rochester 
Rochester,  New  York  14627 


13  abstract 


Suppose  given  k  £  2  normal  populations  ir^. 


.,7t^;  it ^  has  unknown  mean 


and  variance  a2  (i«l,...,k).  We  assume  throughout  that  and  the 

pairings  of  w  [k]  1,l,‘”,T'k  are  completely  unknown  (although  we 

vary  the  distribution  from  normality)  and  consider  the  problem:  e>tlmate 
U  m ,  •  •  •  ,11  based  on  X^,...,X^,  where  X^  is  the  average  of  n  independent 
observations  on  (i  -  l,...,k).  Applications  to  ranking  and  selection 
problems  are  noted. 

X^j,  the  1th  smallest  of  X^,...,X^,  is  a  natural  estimator  of 
(1  <,  i  <,  k)  and  is  studied  with  regard  to  bias,  asymptotic  unbiasedness, 
strong  consistency,  mean  squared  error,  and  minimax  | bias  |  estimator  of  type 
Xj^j  +  a.  Results  for  the  location  parameter  case  are  extended  in  the  normal  cas< 
Maximum  likelihood  estimation,  MLE’s  for  non-1-1  functions,  iterated  MLE’s, 
generalized  MLE's,  and  maximum  probability  estimation  are  studied.  Confidence 
intervals  on  uf.,  (1  4  i  <  k)  are  found  for  location  parameter  populations. 


DD  .’T..  1473 


Unclassified 
Security  Classification 


Security  ClasNificaiion 


I  4 


LINK  A 

'10  4.  if” 1  «l 


.  I..K 

r 


k f y  wonos 


Mathematical  statistics 
Statistical  estimation 
Ordered  parameters 
Ranked  parameters 
Location  parameters 
Ranked  means 

Maximum  likelihood  estimators 

Iterated  maximum  likelihood  estlmatora 

Generalized  maximum  likelihood  estimators 

Maximum  probability  estimators 

Ranking  procedures 

Selection  procedures 

Multiple  decision  procedures 


1  -STRUCTIONS 


1.  ORIGINATING  ACTIVITY  Enter  th*  name  and  a.ldr  -  % 
of  the  i  ontrador,  (ubronlractor.  grant**,  Department  of  f)  - 
frna*  activity  or  other  organization  (corporate  author)  lasuing 
Ih*  report. 

2a.  REPORT  SECURITY  CLASSIFICATION:  Enter  the  over¬ 
all  aecunty  claaalficatlon  of  the  report.  Indicat*  whether 
"Restricted  Data"  ia  included  Marking  is  to  be  in  accord¬ 
ance  with  appropriate  aecurity  regulation*. 

.’ />.  GROUP:  Automatic  downgrading  ia  apecified  in  DoD  Di¬ 
rective  S200.  10  and  Armed  Force*  Industrial  Manual.  Enter 
the  group  number.  Also,  when  applicable,  ahow  that  optional 
marking*  have  been  used  for  Group  3  and  Group  4  aa  author¬ 
ized. 

3.  REPORT  TITLE:  Enter  th*  complete  report  title  in  all 
capital  letter*  Title*  in  all  caaea  should  be  unclassified. 

If  a  meaningful  title  cannot  be  selected  without  classifica¬ 
tion.  ahow  title  classification  in  all  capitals  in  parenthesis 
immediately  following  th*  title. 

4.  DESCRIPTIVE  NOTES  If  appropriate,  enter  the  type  of 
report,  e.g.,  interim,  progress,  aummary,  annual,  or  final. 

Give  thr  inclusive  dates  when  a  specific  reporting  period  ia 
i  overed. 

5.  AUTItOR(S):  Enter  the  namefs)  of  authorfs)  as  shown  on 
or  in  the  report.  Entei  last  name,  first  name,  middle  initial. 

If  T.ilitary,  show  rank  and  branch  of  service.  The  name  of 
the  principal  .  <thor  is  an  absolute  minimum  requirement. 

b.  REPORT  DATL.  Enter  the  date  of  the  report  as  dsy, 
month,  year,  or  month,  year.  If  more  than  one  date  appears 
on  the  report,  use  date  of  publication. 

7 a  TOTAL  NUMBER  OF  PAGES:  The  total  page  count 
should  follow  normal  pagination  procedures,  i.e.,  enter  the 
number  of  page*  containing  information. 

7 h  NUMBER  OF  REFERENCES  Enter  the  total  number  of 
reference*  c  ited  in  th*  report. 

R/>  CONTRACT  OR  GRANT  NUMBER:  If  appropriate,  enter 
the  applicable  number  of  the  contract  or  grant  under  which 
the  report  was  written. 

8fc  Ik.  U  8 d.  PROJECT  NUMBER:  Enter  the  appropriate 
military  department  identification,  such  as  project  number, 
subproject  number,  system  numbers,  task  number,  etc. 

9a  ORIGINATORS  REPORT  NUMBER(S):  Enter  the  offi¬ 
cial  report  number  by  which  th*  document  will  be  identified 
and  controlled  by  the  originating  activity.  This  number  must 
be  unique  to  (Hit.  report. 

9 b  OTHER  REPORT  NUMBER(S):  If  the  report  has  been 
assigned  any  other  report  number*  (either  by  the  originator 
or  6v  the  sponsor),  also  enter  t  hi  a  number(s). 

10.  AVAIL AHIL1TY/LIMITATION  NOTICES:  Enter  any  lim¬ 
itations  on  further  dissemination  of  the  report,  other  than  thoae 


imposed  by  security  classification,  using  standard  statements 
such  as: 

(1)  "Qualified  requesters  may  obtain  copies  of  this 
report  from  DDC." 

(2)  “Foreign  announcement  and  dissemination  of  this 
report  by  DDC  is  not  authorized." 

(3)  "U.  S.  Government  agencies  may  obtain  copies  of 
this  report  directly  from  DDC.  Other  qualified  DDC 
users  shall  request  through 

If 


(4)  "U.  S.  military  agencies  may  obtain  copies  of  this 

report  directly  from  DDC  Other  qualified  users 
shall  request  through 

If 


(5)  "Ail  distribution  of  this  report  is  controlled.  Qual¬ 
ified  DDC  users  shall  request  through 

ii 


If  the  report  has  been  furnished  tc  the  Office  of  Technical 
Services,  Department  of  Commerce,  for  sale  to  the  public,  indi¬ 
cate  this  fact  and  enter  the  price,  if  known. 

11.  SUPPLEMENTARY  NOTES:  Use  for  additional  explana¬ 
tory  notes. 

12.  SPONSORING  MILITARY  ACTIVITY:  Enter  the  name  of 
the  departmental  project  office  or  laboratory  sponsoring  (pay- 
inf  lor)  the  research  and  development.  Include  address. 

13  ABSTRACT:  Enter  an  abstract  giving  a  brief  and  factual 
summary  of  the  document  indicative  of  the  report,  even  though 
it  may  also  appear  elsewhere  in  the  body  of  the  technical  re¬ 
port.  If  additional  space  is  required,  a  continuation  sheet  shall' 
be  attached. 

It  is  highly  desirable  that  the  abstract  of  classified  reports 
be  unclassified.  Each  paragraph  of  the  abstract  shall  end  with 
an  indication  ol  the  military  security  classification  of  thr  in¬ 
formation  in  the  paragraph,  represented  as  (TS).  ( S),  (C).  or  (U) 

There  is  no  limitation  cn  the  length  of  the  abstract.  How¬ 
ever,  the  suggested  length  is  from  ISO  to  22S  words. 

14.  KEY  WORDS:  Key  words  are  technically  meaningful  terms 
or  short  phrsses  that  characterize  a  report  and  may  be  used  as 
index  entries  for  cataloging  the  report.  Key  words  must  be 
selected  so  that  no  security  classification  is  required.  Identi¬ 
fiers,  such  as  equipment  model  designation,  trade  name,  military 
project  code  name,  geographic  location,  may  be  used  as  key 
words  but  will  be  followed  by  an  indication  of  technical  con¬ 
text.  The  assignment  of  links,  rules,  and  weights  is  optional 


•  a'  ise-ysi 


Unclassified _ 

Security  Classification 


